• Sales: 1-800-961-2888
  • Support: 1-800-961-4454

Installing additional munin nodes


Following the article about installing munin monitoring systems, if you want to monitor additional servers you'll need to install a munin node service on each.

What's a munin node?

A munin node is a data collector for munin that runs on a server. A munin master will query the node periodically for its data, and then the master will parse that data and graph it. This article is intended for installation of just a munin node on a different server from the munin master. If you do not yet have a munin master set up, please follow the instructions in the previous article in this series.

Installing the node

On CentOS and Red Hat Enterprise Linux munin is not in the default repository. The easiest way to get the munin node package is to add a repository to your installation that includes it. One such repository is the EPEL repository maintained by the Fedora Core team that contains munin and some other useful enterprise software. Once that repository is added you can install munin from there and include it in future system-wide updates.

To add the EPEL repository to your yum list, run:

sudo rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

With the repository in place, you should be able to install the munin node on your server with:

sudo yum install munin-node

After checking for dependencies yum will ask if you would like to import the GPG key from the EPEL repository. Say yes so yum can verify the authenticity of RPMs retrieved from that site in the future.

To install just a munin node on Debian or Ubuntu, run the commands:

sudo aptitude update
sudo aptitude install munin-node

Configuring the node

Now that the node is installed you'll need to make sure the munin master will be able to communicate with it. The first things you'll need to know are the backend IP addresses of the node server and of the munin master server. You can find IP addresses in the Overview tab for the Cloud Server in the RackConnect Management Interface in the MyRackspace portal or through the Cloud Servers API.

The second IP address listed for a server, the one that starts with "10.", is the backend IP address. If no backend address is listed, contact Rackspace support and they can add one to the server.

Make note of the backend IP addresses of both the node and the munin master. You'll use the backend IP address for munin's communication because that address is not accessible from the rest of the Internet, so it's a little more secure. Even better, traffic on the backend IP addresses does not count toward your servers' bandwidth use for the month (free is good).

Edit munin-node.conf

Note:On debian and ubuntu, Open the /etc/munin/munin-node.conf file and look for this entry:

# Which port to bind to;
host *

Replace the "*" with the backend IP address of the node slice, as in:

# Which port to bind to;
host 10.2.3.4

For all distros, open the /etc/munin/munin-node.conf file and look for this entry:

# A list of addresses that are allowed to connect.  This must be a
# regular expression, since Net::Server does not understand CIDR-style
# network notation unless the perl module Net::CIDR is installed.  You
# may repeat the allow line as many times as you'd like

allow ^127\.0\.0\.1$

That last line is the one you'll want to change. The address has to be in the form of a regular expression because of a perl module used by munin. Fortunately you don't need to know what a regular expression is, or how to write one. Just know that the default expression, "^127.0.0.1$", refers to the IP address "127.0.0.1". You want to change that address to reflect the backend IP address of the munin master server. If your munin master is at "10.33.22.11", for example, you would change that entry to look like:

# A list of addresses that are allowed to connect.  This must be a
# regular expression, since Net::Server does not understand CIDR-style
# network notation unless the perl module Net::CIDR is installed.  You
# may repeat the allow line as many times as you'd like

allow ^10\.33\.22\.11$

Next you'll want to head toward the end of the munin-node.conf file, looking for this entry:

# Which address to bind to;
host *

Replace the "*" with the backend IP address of the node server, as in:

# Which address to bind to;
host 10.2.3.4

Edit the node's iptables

To add the iptables rules you would need to add to allow the munin master to connect to your new node, run the following commands on the node:

For debian and ubuntuIf you have enabled iptables restrictions for your node's slice installation, the iptables rules you would need to add to allow the munin master to connect to your new node look like this:

-I INPUT -p tcp --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPT
-I OUTPUT -p tcp --dport 4949 -m state --state ESTABLISHED -j ACCEPT
sudo /sbin/iptables -I INPUT -p tcp --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo /sbin/iptables -I OUTPUT -p tcp --dport 4949 -m state --state ESTABLISHED -j ACCEPT

For centos, fedora, and RHEL

sudo /sbin/iptables -I INPUT -p tcp --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo /sbin/iptables -I OUTPUT -p tcp --dport 4949 -m state --state ESTABLISHED -j ACCEPT

To save those rules so they'll take effect next time the server is rebooted, run:

sudo /sbin/service iptables save

Restart the node

The munin node wasn't started by the installer, so now that we're done configuring it let's restart that:

sudo /etc/init.d/munin-node start

To make sure the munin-node service starts when your server reboots, you'll also want to make the service active:

sudo /sbin/chkconfig munin-node on

That will complete the setup of the new node server. Next you'll need to configure the master to gather data from the new node.

Adding the node to the master

Connect to the munin master server and edit the /etc/munin/munin.conf file to add the new node to your munin site. In munin.conf look for the "simple host tree" section where you would have set up the master's local node previously.

# a simple host tree
[servername]
    address 127.0.0.1
    use_node_name yes

The easiest way to add the new node to the reporting is to duplicate the host entry you already have, then edit the name and address to match the new node. For example:

# a simple host tree
[servername]
    address 127.0.0.1
    use_node_name yes

[otherservername]
    address 10.2.3.4
    use_node_name yes

Remember not to use a space in the host entry name. Munin's graphing scripts don't seem to like it.

Edit the master's iptables

You will want to add iptables rules on the munin master to be sure it can get out to the new node by running the following commands:

For debian and ubuntu

-I OUTPUT -p tcp --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPT
-I INPUT -p tcp --dport 4949 -m state --state ESTABLISHED -j ACCEPT

For centos, fedora, and RHEL

sudo /sbin/iptables -I OUTPUT -p tcp --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo /sbin/iptables -I INPUT -p tcp --dport 4949 -m state --state ESTABLISHED -j ACCEPT

And to save those rules so they'll take effect next time the server is rebooted, run:

sudo /sbin/service iptables save
See the results

Remember that the munin master runs every five minutes, so it may take that long for the new node to appear on your munin reports. After an appropriate wait visit your main munin page to see if you have a new node listed.

Hopefully you'll see the new node under the entry for your master node, similar to this screenshot:

If you don't see an entry for the new node at all, check the configuration of the munin master (munin.conf on the master server) to make sure the entry was properly added to the host tree.

If you see an entry for the new node but no services are listed in brackets, it means the master knows about the new node but hasn't been able to connect to it. Make sure the node service is running on the node, and if it's running, go back through this guide to double-check the IP addresses in the configurations for the master and node. You might also make sure iptables isn't blocking the master's connection to the node.

If you see the new node and a list of report categories next to it, then you're set. Just remember that it will take a while for munin to collect enough data to make any noticeable graphs for the new node.

Summary

Adding a node to a munin master boils down to configuring and starting a munin-node service on the new node, then telling the munin master about it. If you have more than one node to add, go through this guide for each additional node to get them onto the munin reports too. With that done you'll have a single page you can go to that will link to reports for all your servers. Convenient!

The next article in this series will discuss additional service plug-ins for munin, how to tell munin nodes to use the ones you need, and how to install new plug-ins on nodes.

-- Jered







© 2011-2013 Rackspace US, Inc.

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License


See license specifics and DISCLAIMER