Unix TA clarity - permissions and readme file

[msarro]

Hi Everyone, I am working with the Unix TA and am trying to figure out some issues. When I start the splunk forwarder, I see the following messages showing up:

04-19-2013 15:26:38.665 +0000 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/PE_Splunk_TA_nix/bin/protocol.sh" /bin/sh: /opt/splunkforwarder/etc/apps/PE_Splunk_TA_nix/bin/protocol.sh: Permission denied
04-19-2013 15:26:38.666 +0000 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/PE_Splunk_TA_nix/bin/vmstat.sh" /bin/sh: /opt/splunkforwarder/etc/apps/PE_Splunk_TA_nix/bin/vmstat.sh: Permission denied
04-19-2013 15:26:38.666 +0000 INFO  ExecProcessor - Ran script: /opt/splunkforwarder/etc/apps/PE_Splunk_TA_nix/bin/protocol.sh, took 6.084 milliseconds to run, 0 bytes read, exited with code 126
04-19-2013 15:26:38.666 +0000 INFO  ExecProcessor - Ran script: /opt/splunkforwarder/etc/apps/PE_Splunk_TA_nix/bin/top.sh, took 3.046 milliseconds to run, 0 bytes read, exited with code 126
04-19-2013 15:26:38.677 +0000 INFO  ExecProcessor - Ran script: /opt/splunkforwarder/etc/apps/PE_Splunk_TA_nix/bin/vmstat.sh, took 11.80 milliseconds to run, 0 bytes read, exited with code 126

No where in any of the documentation does it mention permission settings. The directory is created as the root user, does it need to be changed to splunk? Does the splunk user have the ability to run all of the scripts? Based on another thread I found, I have set the app to belong to the splunk user/group.

Also, inside the TA app itself the readme file states the following:

Using this Technology Add-on:
----------------------------------------
    Configuration: Manual
    Ports for automatic configuration: None
    Scripted input setup: Not applicable

    The source type must be assigned according to the platform that generated the data. See the table below:

    ______________________________________________________________________
    | Source type  | Associated Platform                                 |
    |--------------|-----------------------------------------------------|
    | dhcpd        | Use for logs from the dhcpd daemon                  |
    | aix_secure   | Use for systems logs from AIX                       |
    | osx_secure   | Use for systems logs from Macintosh OS-X            |
    | linux_secure | Use for systems logs from Red-hat based Linux hosts |
    | syslog       | Use for systems logs from Debian based Linux hosts  |
    |______________|_____________________________________________________|

However none of the documentation located here states anything about such configurations. I can't seem to find the place where the configuration needs to be done. If anyone can help with these two issues I'd appreciate it.

[fharding]

I'm necromancing this question. You said you ran chmod +x *.sh on the app's bin dir, but not sure if you ran it on the app's bin dir on the host with the forwarder installed, or the app's bin dir in your deployment-apps on the deployment server, so I'll post this in case someone finds it useful. I would imagine running chmod on the scripts in the app directory on the host with the forwarder installed would allow the files to get overwritten when a change is detected and the deployment server re-pushes the old files that don't have the execute bit set.

Anyway, I had the same issue over here. My solution was the same as yours I think.

On your forwarder, check out the scripts directory that has been pushed down from the deployment server:

cd /opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin
ls -lh

Will probably look something like this:

-rw------- 1 splunk splunk 3.6K Jun 26 23:39 common.sh
-rw------- 1 splunk splunk 3.5K Jun 26 23:39 cpu.sh
-rw------- 1 splunk splunk 4.9K Jun 26 23:39 df.sh
...

The files do not have execute permission by the splunk user.

On your deployment server, you need to give the scripts execute permission, then reload deploy-server:

cd /opt/splunk/etc/deployment-apps/Splunk_TA_nix/bin
ls -lh

Same deal - will probably look something like this:

-rw------- 1 splunk splunk 3.6K Jun 26 23:39 common.sh
-rw------- 1 splunk splunk 3.5K Jun 26 23:39 cpu.sh
-rw------- 1 splunk splunk 4.9K Jun 26 23:39 df.sh
...

Fix the permissions:

chmod u+x *.sh

Reload the deployment server:

/opt/splunk/bin/splunk reload deploy-server

Wait for the files to push down.
You could run this on the forwarder and watch the date on the Splunk_TA_nix dir:

watch -n 1 'ls -lh /opt/splunkforwarder/etc/apps'

It should push down eventually.

Now, on your search head, run this over the last 60 minutes or so to look for results from one of the check scripts:

index=main host="YOUR-HOST" eventtype=cpu

If you don't see anything, wait a few minutes and run the search again.

You could also run this splunk search to see if you are still having the errors:

index=_internal host="YOUR-HOST" "ERROR ExecProcessor"

Regarding the ifconfig error you were having, try this on the host with the forwarder installed:

On some distros (debian), non-root accounts don't know the path to ifconfig.
Switch to splunk user:

sudo su - splunk
which ifconfig

(might return nothing)

ifconfig

Might return:

-bash: ifconfig: command not found

It's probably in sbin

/sbin/ifconfig

If that is the case, maybe you could add something in the pushed nix app that lets it know about that path, or calls ifconfig directly.

[fharding]

Weird. The last part of that got screwed up. Should read:

Regarding the ifconfig error you were having, try this on the host with the forwarder installed.

On some distros (debian), non-root accounts don't know the path to ifconfig.

Switch to splunk user:
sudo su - splunk
which ifconfig

Might return nothing.
ifconfig

Might return:
-bash: ifconfig: command not found

It's probably in sbin.
/sbin/ifconfig

If that is the case, maybe you could add something in the pushed nix app that lets it know about that path, or calls ifconfig directly.

[msarro]

First issue appears to be partly solved - in the app's bin directory, ran chmod +x *.sh to allow all items to be executed. Still receiving an error for interfaces.sh, stating that ifconfig isn't found.