PNP4Nagios with SElinux on CentOS / RHEL 6

PNP4Nagios is commonly used to add performance graphs to a Nagios installation.

For additional security, SElinux is enabled on the monitoring host. There is no standard SElinux policy for applications like PNP4Nagios, so we need to develop a custom policy. This sounds harder than it actually is:

  • Run the software as you normally would (SElinux will interfere, so prepare for errors)
  • Extract audit messages and use them to create or update a local SElinux policy for the software
  • Repeat until everything works

In this example, I am running Nagios 3.2.3 with PNP4Nagios 0.6.16 on EL6, 64-bit.

After configuring Nagios and PNP4Nagios integration in Synchronous Mode (see documentation), I noticed that PNP4Nagios is not logging any performance data to /var/lib/pnp4nagios/.

Normally, PNP4Nagios should automatically create directories and files under /var/lib/pnp4nagios as performance data is received by Nagios. This smells of an SElinux issue, so check /var/log/audit/audit.log for suspicious messages. Sure enough, several audit messages have been logged. They look like this:

type=AVC msg=audit(1329129875.344:198212): avc:  denied  { getattr } for  pid=26692 comm="process_perfdat" \
    path="/var/lib/pnp4nagios/orac/Root_Partition.xml.26692" dev=dm-0 ino=1444378 \
    scontext=unconfined_u:system_r:nagios_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file
type=SYSCALL msg=audit(1329129875.344:198212): arch=c000003e syscall=5 success=no exit=-13 a0=3 a1=25440a0 \
    a2=25440a0 a3=0 items=0 ppid=26691 pid=26692 auid=0 uid=498 gid=498 euid=498 suid=498 fsuid=498 egid=498 \
    sgid=498 fsgid=498 tty=(none) ses=14942 comm="process_perfdat" exe="/usr/bin/perl" subj=unconfined_u:system_r:nagios_t:s0 key=(null)

Create a policy

You can run the “audit2allow” command (part of the policycoreutils-python RPM) to display suggested policy improvements based on the audit log:

audit2allow -a

The output can be saved in a file, for example local_nagios.te:

grep nagios_t /var/log/audit/audit.log | audit2allow -l -v -m local_nagios > local_nagios.te

This generates an output file suitable for compiling into a custom SElinux module.

Note: ALWAYS prefix the policy name with something like local_ to prevent overwriting system policies!

Test and refine the policy

Compile and load the SElinux policy module:

checkmodule -M -m -o local_nagios.mod local_nagios.te
semodule_package -o local_nagios.pp -m local_nagios.mod
semodule -v -i local_nagios.pp

Note: The above tools can be found in the checkpolicy and policycoreutils RPMs.

Re-run the software and check for SElinux audit messages. New issues can be captured and translated into a new policy:

grep nagios_t /var/log/audit/audit.log | audit2allow -l -v -m local_nagios > local_nagios.te_NEW

Merge the new results (in local_nagios.te_NEW) with your existing policy (in local_nagios.te). Compile and reload the module.

Lather, rinse, repeat ;-)

Results

After some iterations, your local_nagios.te file will look something like this:

module local_nagios 1.0;

require {
    type nagios_t;
    type var_log_t;
    type var_lib_t;
    class dir { write create add_name remove_name };
    class file { create getattr ioctl lock open read rename unlink write };
}

#============= nagios_t ==============
allow nagios_t var_lib_t:dir { add_name create remove_name write };
allow nagios_t var_lib_t:file { create getattr ioctl lock open read rename unlink write };
allow nagios_t var_log_t:file { read rename unlink };

If all is well, the audit.log should not show any new messages for nagios_t:

clear;tail -f /var/log/audit/audit.log |grep nagios_t

Note: The new SElinux policy will survive reboots; it is automatically copied to /etc/selinux/targeted/modules/active/modules/local_nagios.pp.

Enjoy!

Workaround for Nagios check_disk failure in RHEL / CentOS 6.2

After updating from EL 6.1 to 6.2, the Nagios “check_disk” plugin suddenly stopped working with “Permission denied” errors. This problem is related to the SElinux policy (you *are* running with SElinux enabled, aren’t you?).

By default, these AVC denials are not logged in /var/log/audit/audit.log which makes this problem harder to spot (if you want, you can enable all audit-messages by running semodule -DB).

There are at least two relevant entries in Bugzilla:

  • Bug 771245 – nagios-plugins-disk fails when checking /boot on RHEL6.2 boxes
  • Bug 768055 – SELinux silent denials of Nagios NRPE check of /boot

Fortunately, there is a simple workaround while we wait for an updated selinux-policy package. As root, do the following:

chcon -t nagios_unconfined_plugin_exec_t /usr/lib64/nagios/plugins/check_disk

Or, for 32-bit systems:

chcon -t nagios_unconfined_plugin_exec_t /usr/lib/nagios/plugins/check_disk

No need to restart anything; just wait until Nagios re-checks the service and the problem should be gone. Enjoy!

Controlling the Foscam FI8919W IP Camera

Foscam FI8918W IP Camera, whiteThe Foscam FI8919W Pan-Tilt camera supports a number of preset locations, or Presets.

The onboard web interface does not seem to offer a way to configure these presets – but there is another way!

I have written a couple of small shell scripts (for Linux or Mac OS X) that allow you to set a preset, move to a preset and even take a snapshot right from the command line.

The first script stores the current camera position into the specified preset.

You need to open a web browser or camera app (my favorite: Live Cams Pro on iPad/iPhone) and set the camera position.

Then, run this script, specifying the preset number (for example, “foscam_set 0” to set the first preset):

#!/bin/bash
# Store current camera position in specified preset (0..16)

# Commandline handling
#
preset=$1
if [ -z $preset ];
then
    echo "Syntax: $0 <preset>, where preset is a number (0..16)"
    exit 1
fi

# Address (or address:port number) where to reach the camera
# Username / password to access camera functions
#
CAMERA=192.168.1.2
USERNAME=theUsername
PASSWORD=thePassword

# Presets are set using URL with command (30 + preset*2), and recalled using URL with command (31 + preset*2)
#
command=$((30 + 2*$preset))
echo "Storing current camera position in preset ${preset} (command = ${command})"
wget -O - http://${CAMERA}/decoder_control.cgi?command=${command}\&user=${USERNAME}\&pwd=${PASSWORD}

Next, a script to move the camera to a specified preset (for example, “foscam_go 3“:

#!/bin/bash
# Move camera to specified preset (0..16)

# Commandline handling
#
preset=$1
if [ -z $preset ];
then
    echo "Syntax: $0 <preset>, where preset is a number (0..16)"
    exit 1
fi

# Address (or address:port number) where to reach the camera
# Username / password to access camera functions
#
CAMERA=192.168.1.2
USERNAME=theUsername
PASSWORD=thePassword

# Presets are set using URL with command (30 + preset*2), and recalled using URL with command (31 + preset*2)
#
command=$((31 + 2*$preset))
echo "Moving camera to preset ${preset} (command = ${command})"
wget -O - http://${CAMERA}/decoder_control.cgi?command=${command}\&user=${USERNAME}\&pwd=${PASSWORD}

Finally, a demo script that moves the camera to a preset, takes a snapshot and stores it locally (based on a post by 1994MGoBlue):

#!/bin/bash
# Take snapshots of certain preset camera locations

# Address (or address:port number) where to reach the camera
# Username / password to access camera functions
#
CAMERA=192.168.1.2
USERNAME=theUsername
PASSWORD=thePassword

# The camera should normally be in this position
DEFAULT_PRESET=3

# Seconds to sleep after issuing a camera move command, allow it to reach new position.
# You may have to change this value to your needs
DELAY=10

# Presets are set using URL with command (30 + preset*2), and recalled using URL with command (31 + preset*2)
#
for preset in 0 1 2 3 4;
do
    command=$((31 + 2*$preset))
    echo "Taking snapshot in preset ${preset} (command = ${command})"

    # Move camera, delay, take snapshot (stored in /tmp/)
    wget -O - http://${CAMERA}/decoder_control.cgi?command=${command}\&user=${USERNAME}\&pwd=${PASSWORD}
    sleep ${DELAY}
    wget -O /tmp/preset-${preset}.jpg http://${CAMERA}/snapshot.cgi?user=${USERNAME}\&pwd=${PASSWORD}
done

# Done, send camera back to default position
command=$((31 + 2*$DEFAULT_PRESET))
wget -O - http://${CAMERA}/decoder_control.cgi?command=${command}\&user=${USERNAME}\&pwd=${PASSWORD}

Enjoy!

Making Cobbler work with SElinux on CentOS / RHEL 6

By default, Cobbler will not work properly on a CentOS / RHEL 6 machine with SElinux enabled. The easy way out is to disable SElinux entirely, but I prefer to write a custom policy instead – it is not that difficult.

The basic approach is this:

  1. Use Cobbler as you normally would (you will trigger several SElinux denials, so expect errors)
  2. Extract the relevant SElinux audit messages; convert them into a local policy
  3. Load your local policy
  4. Repeat steps 1..3 until everything works as expected

First attempt: the “cobbler import” command fails; rsync cannot access files on the mounted DVD ISO. Time to start writing a local policy!

The following command generates a basic SElinux policy from the SElinux audit messages:

  cat /var/log/audit/audit.log | audit2allow -l -v -m local > local.te

The resulting local.te file (ASCII, open it in your favorite editor) will list various items, some if which are not related to the Cobbler / Rsync operations. Edit the file to taste. Now, compile and load that policy:

  checkmodule -M -m -o local.mod local.te
  semodule_package -o local.pp -m local.mod
  semodule -v -i local.pp

Note: every invocation of “audit2allow -l” will overwrite your local.te policy file with new events since the last time a policy module was loaded. This is why you should keep backup copies of the previous versions so you can merge new events in with the existing ones.

In the end, you will end up with a policy in local.te like this:

module local 1.0;

require {
    type cobblerd_t;
    type cobbler_var_lib_t;
    type iso9660_t;
    type public_content_t;
    type rpm_var_lib_t;
    type rsync_etc_t;
    type security_t;
    type tmp_t;
    class capability { sys_module fsetid };
    class dir { add_name create getattr open read remove_name rmdir search write };
    class file { create getattr open read unlink write };
    class lnk_file create;
    class unix_dgram_socket create;
}

#============= cobblerd_t ==============
allow cobblerd_t cobbler_var_lib_t:lnk_file create;
allow cobblerd_t iso9660_t:dir { open read search getattr };
allow cobblerd_t iso9660_t:file { open read getattr };
allow cobblerd_t public_content_t:dir { write rmdir remove_name };
allow cobblerd_t rpm_var_lib_t:dir { open read search getattr write };
allow cobblerd_t rsync_etc_t:file create;
allow cobblerd_t security_t:dir read;
allow cobblerd_t self:capability fsetid;
allow cobblerd_t self:unix_dgram_socket create;
allow cobblerd_t tmp_t:dir { add_name create remove_name rmdir write };
allow cobblerd_t tmp_t:file { create getattr open read unlink write };

Panic after installing new kernel on CentOS 6

I just finished updating all of my CentOS 6 machines, resulting in a couple of machines (VMs) no longer booting:

Kernel panic - not syncing: No init found. Try passing init= option to kernel.

It appears that “yum update” installs the new kernel:

[root@c3p0 boot]# rpm -qa |grep ^kernel-
kernel-2.6.32-131.17.1.el6.x86_64
kernel-firmware-2.6.32-131.17.1.el6.noarch
kernel-firmware-2.6.32-71.29.1.el6.noarch
kernel-2.6.32-71.29.1.el6.x86_64

… but no initramfs is created under /boot:

[root@c3p0 boot]# ls -l /boot
total 25561
-rw-r--r--. 1 root root 100203 Oct 6 20:44 config-2.6.32-131.17.1.el6.x86_64
-rw-r--r--. 1 root root 97911 Jun 27 21:08 config-2.6.32-71.29.1.el6.x86_64
drwxr-xr-x. 3 root root 1024 Aug 16 12:58 efi
drwxr-xr-x. 2 root root 1024 Dec 1 17:15 grub
-rw-r--r--. 1 root root 13435614 Aug 16 12:59 initramfs-2.6.32-71.29.1.el6.x86_64.img
drwx------. 2 root root 12288 Aug 16 12:54 lost+found
-rw-r--r--. 1 root root 165827 Oct 6 20:47 symvers-2.6.32-131.17.1.el6.x86_64.gz
-rw-r--r--. 1 root root 160602 Jun 27 21:11 symvers-2.6.32-71.29.1.el6.x86_64.gz
-rw-r--r--. 1 root root 2279162 Oct 6 20:44 System.map-2.6.32-131.17.1.el6.x86_64
-rw-r--r--. 1 root root 2228188 Jun 27 21:08 System.map-2.6.32-71.29.1.el6.x86_64
-rwxr-xr-x. 1 root root 3882992 Oct 6 20:44 vmlinuz-2.6.32-131.17.1.el6.x86_64
-rwxr-xr-x. 1 root root 3795744 Jun 27 21:08 vmlinuz-2.6.32-71.29.1.el6.x86_64

Also, in /etc/boot/grub.conf, the initrd entry is missing for the new kernel:

[root@c3p0 boot]# cat /etc/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/mapper/VolGroup-lv_root
#          initrd /initrd-[generic-]version.img
#boot=/dev/vda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS Linux (2.6.32-131.17.1.el6.x86_64)
	root (hd0,0)
	kernel /vmlinuz-2.6.32-131.17.1.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_LVM_LV=VolGroup/lv_root rd_LVM_LV=VolGroup/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet
title CentOS (2.6.32-71.29.1.el6.x86_64)
	root (hd0,0)
	kernel /vmlinuz-2.6.32-71.29.1.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_LVM_LV=VolGroup/lv_root rd_LVM_LV=VolGroup/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet
	initrd /initramfs-2.6.32-71.29.1.el6.x86_64.img

I remember seeing this problem before on a fairly recent Fedora system, probably Fedora 14 or 15, but never on older CentOS / RHEL releases.

My guess is that this is a problem with Dracut. You can re-generate the initramfs by running dracut (as root, of course), but I’d rather not have to perform emergency surgery on lots of (virtual) machines…

Anyway, the command line would look like this:

dracut initramfs-${version}.img ${version}
dracut initramfs-2.6.32-131.17.1.el6.x86_64.img 2.6.32-131.17.1.el6.x86_64