Daniel Watrous on Software Engineering

A Collection of Software Problems and Solutions

Software Engineering

Increase Munin Resolution to One Minute

I’ve recently been conducting some performance testing of a PaaS solution. In an effot to capture specific details resulting from these performance tests and how the test systems hold up under load, I looked into a few monitoring tools. Munin caught my eye as being easy to setup and having a large set of data available out of the box. The plugin architecture also made it attractive.

One major drawback to Munin was the five minute resolution. During performance tests, it’s necessary to capture data much more frequently. With the latest Munin, it’s possible to easily get down to one minute resolution with only a couple of minor changes.

One minute sampling

Since Munin runs on a cron job, the first step to increase the sampling resolution is to run the cron job more frequently. There are two cron jobs to consider.

/etc/cron.d/munin-node

the /etc/cron.d/munin-node cron runs the plugins that actually generate datapoints and store them in RRD files. In order to increase the amount of data collected, modify this file to sample more frequently (up to once a minute).

/etc/cron.d/munin

The /etc/cron.d/munin cron runs the Munin core process which builds the HTML and generates the graphs. Running this cron more frequently will not increase the number of data samples collected. However, if you do modify the munin-node cron to collect more samples, you may want to update this cron so that data is visible as it is collected.

Change plugin config to capture more data

The plugin config must be modified to include (or change) two arguments.

  • update_rate: this modifies the step for the rrd file
  • graph_data_size: this sets the rrd file size to accommodate the desired amount of data

To sample at a rate of once per minute, the config would need to return the following along with other values.

graph_data_size custom 1d, 1m for 1w, 5m for 1t, 15m for 1y
update_rate 60

Historical Data

After changing the rrd step using update_rate and redefining how much data to capture using graph_data_size, the existing rrd file will no longer accommodate the new data. If the historical data is not needed, the rrd files can be deleted from /var/lib/munin/<domain>. New files with the desired step and amount of data will be created when update is run next.

If the historical data is needed, it may be possible to reconfigure the existing data files, but I haven’t personally tried this.

4 Comments Increase Munin Resolution to One Minute

  1. nicolas R

    Hi,

    thanks for this article.

    But I have a question :

    For me it’s not clear where to change update_rate and graph_data_size

    I have to say that a read lot of post/blog/faq that talk about this but no one says clearly example : “edit this file /usr/lib/munin/plugins/whatever_plugins and add these two lines at the beginning or at the end or at this place”

    regards.

    Reply
  2. Danish Amjad

    Hi,

    Thanks for the article. I followed the settings as mentioned in the article. But the graph I get is discontinuous. For continuity, I changed the Munin Cron to update every 2 minutes. Is it the right way to solve discontinuity.

    Reply

Leave A Comment