Splunk for Nagios not working although log files are indexed

[kuramanga]

I followed the guide on the main page for this app, installed SplunkForNagios, added an index called nagios and configured Splunk Universal Forwarder to forward /var/log/nagios3/nagios.log as well as hostperf and serviceperf files.

Within Splunk I can see these files are available and can look within them, Nagios shows several warnings and critical alerts but Splunk For Nagios shows 0 warning and 0 critical.

If I switch to the search app and search within /var/log/nagios3/nagios.log for SERVICE ALERT I can see many.

Have I missed something or screwed up somewhere? I've noticed that the nagios index is 1MB with 0 entries in it.

Apologies, I'm new to Splunk

[kuramanga]

I am a complete idiot, I had index=nagios on the Splunk server not on the Nagios forwarder.

[lukeh]

It may be possible that the data is being indexed in the default index called main, instead of the required destination index called nagios.

If you run the following search in Splunk for Nagios are there any results:

index=nagios

If not, try the following search and note the name of the index listed in the field picker at left:

source=/var/log/nagios3/nagios.log 

If the results are not in the nagios index, please edit $SPLUNK_HOME/etc/apps/search/local/inputs.conf on the Nagios server and add the following key/value pair:

index = nagios

then restart the Splunk UF agent:

./splunk restart

Note: please ensure that the nagios index is owned by the splunk user, if not please fix the ownership and restart splunk on your splunk server.

All the best,

Luke 🙂

[kuramanga]

Hey Luke, thanks for the reply.

You're correct, the nagios data is indexed in the main index, I have previously created $SPLUNK_HOME/etc/apps/search/local/inputs.conf (it did not exist) and it already has the following content:

$ cat /opt/splunk/etc/apps/search/local/inputs.conf
index = nagios

The nagios index is now owned by the SplunkForNagios app (it seems splunk gives permission to the index to the app you were previously viewing when you create that index) but none of these changes are filling the nagios index with data.