NMON fifo leaking ?

[agneticdk]

Hi all

I would like to know if any have problems with leakage of fifo processes started by the NMON add on, on the servers running the forwarders. It is the latest version, and a debian 7.11 (kernel 3.2.0-4-amd64)

We see some (not all) servers leaking processes (this is only a small part, hundreds of these processes are running):

0 S root 4578 1 0 80 0 - 5534 ? 2018 ? 00:00:13 python /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.py --fifo fifo1
0 S root 4580 4578 0 80 0 - 1046 ? 2018 ? 00:00:00 /bin/sh -c /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.sh /opt/splunkforwarder/var/log/nmon/var/nmon_repository/fifo1/nmon.fifo
0 S root 4581 4580 0 80 0 - 1046 ? 2018 ? 00:00:05 /bin/sh /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.sh /opt/splunkforwarder/var/log/nmon/var/nmon_repository/fifo1/nmon.fifo
0 S root 4738 1 0 80 0 - 5534 ? Jan24 ? 00:00:03 python /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.py --fifo fifo1
0 S root 4740 4738 0 80 0 - 1046 ? Jan24 ? 00:00:00 /bin/sh -c /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.sh /opt/splunkforwarder/var/log/nmon/var/nmon_repository/fifo1/nmon.fifo
0 S root 4741 4740 0 80 0 - 1046 ? Jan24 ? 00:00:01 /bin/sh /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.sh /opt/splunkforwarder/var/log/nmon/var/nmon_repository/fifo1/nmon.fifo
0 S root 4953 1 0 80 0 - 5534 ? Mar06 ? 00:00:01 python /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.py --fifo fifo1
0 S root 4955 4953 0 80 0 - 1046 ? Mar06 ? 00:00:00 /bin/sh -c /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.sh /opt/splunkforwarder/var/log/nmon/var/nmon_repository/fifo1/nmon.fifo
0 S root 4956 4955 0 80 0 - 1046 ? Mar06 ? 00:00:00 /bin/sh /opt/splunkforwarder/etc/apps/TA-nmon/bin/fifo_reader.sh /opt/splunkforwarder/var/log/nmon/var/nmon_repository/fifo1/nmon.fifo

As I can see, it is init that is parent to the processes, so it looks to me that the splunk framework loses control of these fifo processes, and so they do not terminate.

Any others having these issues ? Any knows of this problem, or some pointers to what can be done to eliminate these problems ?

Thank you
André

[jessec_splunk · Accepted solution]

The TA-NMON has a troubleshooting guide here.

I would also (if you haven't done so), upgrade TA-NMON to the latest 1.3.33 version released on April 10, 2018.

[jessec_splunk]

Not sure why link didn't show up. Here is the troubleshooting guide link:

https://ta-nmon.readthedocs.io/en/latest/troubleshoot.html?highlight=fifo

[agneticdk]

Thank you. We are running 1.3.33, sorry I missed that.
The guide say that only one fifo_reader.py should be running, and that: "You will observe that killing the nmon process will automatically terminate the fifo_reader.py", which does not. I can see that it then states "If the processes do not stop, then your problem became mine and please open an issue" - I will do that 🙂