I've installed the Splunk App for Unix and Linux on a Windows 2008 Server and Splunk Add-on for Unix and Linux on a Universal Forwarder on a Linux RedHat 5.5 Server.
I've configured the Splunk App for Unix and Linux as shown in the documentation and activated scripted inputs on the forwarder. The forwarder is sending the data correctly (I find them in the "os" index).
I've configured the Alert "Processes_Exceeds_by_Host" with a threshold of 10 processes, but I can't find any alerts in the Alerts dashboard of Splunk App for Unix and Linux.
What configuration I have missed?
I fought with the same issue and solved it. Same setup. 1 Splunk Server with 1 Splunk Forwarder. The issue is the way the saved searches are configured in the Splunk App for Unix and Linux. Below is the relevant stanza in its default configuration.
[Processes_Exceeds_by_Host] action.summary_index = 1 action.summary_index.marker = unix_aggregated_alerts action.summary_index._name = unix_summary alert.digest_mode = True alert.expires = 1d alert.suppress = 1 alert.suppress.period = 1m alert.track = 1 auto_summarize.dispatch.earliest_time = -5m@m cron_schedule = */5 * * * * counttype = number of events disabled = 1 dispatch.earliest_time = -5m@m dispatch.latest_time = now displayview = unix_flashtimeline enableSched = 1 quantity = 1 relation = greater than search = `Processes_Exceeds_by_Host("`_unix_alert_threshold_Processes_Exceeds_by_Host`")`
The issues is the combination of the quantity and relation setting. In plain english it's saying "if # of results returned is greater than 1 fire the alert". When you have ONE server it will never fire. Also, it would hide the fact that there is a server firing alerts.
I know this is an old question, hopefully it helps someone else.
@fenrisdacat I have followed the same steps as you mentioned but still not getting the alerts on Alerts Dashboard it is showing as I have added "quantity = 0" for each stanza but not getting alerts.
@fenrisdacat, after making the changes in SPLUNK_HOME/etc/apps/splunk_app_for_nix/local/savedsearches.conf, do you leave the file in that location or do you move it back to SPLUNK_HOME/etc/apps/splunk_app_for_nix/default location? I just installed the app this week and running into the same issue. Thanks
I encountered same problem. To me about
Host were OK. But for
Alerts part. I set that in Setting with some threshold.
And then I pointed to back to
Alerts part. Just thing I see is
Alert not found