Splunk App for Unix and Linux - Alerts don't fire

[apaz_71]

Hi,

I've installed the Splunk App for Unix and Linux on a Windows 2008 Server and Splunk Add-on for Unix and Linux on a Universal Forwarder on a Linux RedHat 5.5 Server.

I've configured the Splunk App for Unix and Linux as shown in the documentation and activated scripted inputs on the forwarder. The forwarder is sending the data correctly (I find them in the "os" index).

I've configured the Alert "Processes_Exceeds_by_Host" with a threshold of 10 processes, but I can't find any alerts in the Alerts dashboard of Splunk App for Unix and Linux.

What configuration I have missed?

Thanks

Apaz

[khilanm]

• Actually, I have also faced the same issue the reason is that alerts are working but not satisfied condition to triggered activity.
• So Do one thing that in Unix app -> settings -> alerts enable all alerts and make threshold as minimum as possible.
• Then wait for some time and in alerts dashboard you can see alerts.
• Or in Activity -> Triggered alerts you can see alerts are triggered.

[jackjack]

How long did it take for alerts to show up in the dashboard for you? I see them firing if I go to Activity > Triggered Alerts but I still do not see them in my Splunk App for Unix Alerts Dashboard.

[fenrisdacat]

Hi,

I fought with the same issue and solved it. Same setup. 1 Splunk Server with 1 Splunk Forwarder. The issue is the way the saved searches are configured in the Splunk App for Unix and Linux. Below is the relevant stanza in its default configuration.

[Processes_Exceeds_by_Host]
action.summary_index = 1
action.summary_index.marker = unix_aggregated_alerts
action.summary_index._name = unix_summary
alert.digest_mode = True
alert.expires = 1d
alert.suppress = 1
alert.suppress.period = 1m
alert.track = 1
auto_summarize.dispatch.earliest_time = -5m@m
cron_schedule = */5 * * * *
counttype = number of events
disabled = 1
dispatch.earliest_time = -5m@m
dispatch.latest_time = now
displayview = unix_flashtimeline
enableSched = 1
quantity = 1
relation = greater than
search = `Processes_Exceeds_by_Host("`_unix_alert_threshold_Processes_Exceeds_by_Host`")`

The issues is the combination of the quantity and relation setting. In plain english it's saying "if # of results returned is greater than 1 fire the alert". When you have ONE server it will never fire. Also, it would hide the fact that there is a server firing alerts.

The fix:

1. copy $SPLUNK_HOME/etc/apps/splunk_app_for_nix/default/savedsearches.conf to $SPLUNK_HOME/etc/apps/splunk_app_for_nix/local/savedsearches.conf
2. Edit $SPLUNK_HOME/etc/apps/splunk_app_for_nix/local/savedsearches.conf
3. Change quantity under [Processes_Exceeds_by_Host] to 0

I know this is an old question, hopefully it helps someone else.

-Felix

[dhihoriya_splun]

@fenrisdacat I have followed the same steps as you mentioned but still not getting the alerts on Alerts Dashboard it is showing as I have added "quantity = 0" for each stanza but not getting alerts.
Thanks

[dharveynswccd]

@fenrisdacat, after making the changes in SPLUNK_HOME/etc/apps/splunk_app_for_nix/local/savedsearches.conf, do you leave the file in that location or do you move it back to SPLUNK_HOME/etc/apps/splunk_app_for_nix/default location? I just installed the app this week and running into the same issue. Thanks

[luanvn]

I encountered same problem. To me about Home, Metric, Host were OK. But for Alerts part. I set that in Setting with some threshold.
And then I pointed to back to Alerts part. Just thing I see is Alert not found