Splunk App and Add-on for Unix and Linux: Why are all servers showing "unknown" for vmstat.sh?

[dperry]

I must be missing something here......I have several AIX 6.1 & RHEL 5/6 servers installed with the Add on Splunk_TA.

On the Splunk App for Unix, all of the servers do not show the output for the command vmstat.sh, instead it says unknown - is vmstat.sh enabled?

I look at the Add-on configuration and it's enabled.
When I go to the location: /opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin vmstat.sh I can run it as root, but when I run it as Splunk, nothing comes up. The permission for all the other commands are the same and they do work: cpu.sh, df.sh,ps.sh annd so on.

Any thing I can look at to see if the vmstat.sh command is not configured right?

[andygerber]

On 16.06 Ubuntu for example, sysstat is not installed by default, which is required for vmstat.sh and cpu.sh to work. Do an apt-get install sysstat and things will work much better.

[dperry · Accepted solution]

After further investigation I figured it out. Permissions, Permissions, Permissions. Although there were no indication from the Splunkd logs that the command vmstat.sh was failing I opened the Splunk_TA_nix/bin/vmstat.sh in VI:

elif [ "x$KERNEL" = "xAIX" ] ; then
set -x
assertHaveCommand uptime
assertHaveCommand ps
assertHaveCommand vmstat
assertHaveCommandGivenPath /usr/sbin/swap
assertHaveCommandGivenPath /usr/bin/svmon
CMD='eval uptime ; ps -e | wc -l ; ps -em | wc -l ; /usr/sbin/swap -s ; vmstat 1 1 ; vmstat -s ; svmon'

I placed a set -x to see what the script was doing and it was stopping at least two commands...The Splunk user account did not have the correct permissions to run these. I added Splunk in the system group and it started to collect the data.

[divyavikas123]

I placed set -x in vmstat , its showing only cpu,memory and disk but its not showing RAM value,please tell me if I need to modify any more things.