Alerting

How to limit alert to only one line?

zliu
Splunk Employee
Splunk Employee

Alert was triggered because of: 'Saved Search [fortyfor-test]: number of events(2)'

Apr 26 20:59:15 dist puppetd[1534]: [ID 702911 daemon.notice] Finished catalog run in 3.48 seconds
Apr 26 20:51:31 [10.1.16.43.239.62] puppetd[1783]: [ID 702911 daemon.notice] Finished catalog run in 2.99 seconds

How to limit alert to only one line?

set in alert_actions.conf

maxresults = 1


[email]
maxresults = 1

on Splunk 4.1.0.

Tags (1)

sideview
SplunkTrust
SplunkTrust

Maybe some more details are missing but it sounds like what you really want is for the search running on a schedule to have either stats command or a head command on the end of it.

eg:

<your search> | head 1
<your search> | stats count
<your search> | stats count dc(your_extracted_process_id_field) avg(your_extracted_seconds_field) 

that way there's just only one result in the search to send period.

You can have one saved search that is for users to run from the menus, that doesnt have this stats command, and then have a different one scheduled, (that can be marked as not visible from the UI).

sideview
SplunkTrust
SplunkTrust

This is a common request, and to paraphrase it and make sure i understand, it boils down to:

"I want to check the alert every 5 minutes but once it starts failing I only want it to email me like once every 4 hours or something"

This (and much more) is being developed for our next big release. However as Simeon says there are probably ways to do it even today, and in the interim we might consider testing out and writing up some of the less half-baked ideas somewhere.

This is one half-baked idea, but it's possible to set this up yourself with only one custom shell script that would run a splunk search, and with the rest done in the UI.

1) Aforementioned custom shell script gets hooked up in the "Trigger Shell Script" section of the alert. It uses a search like the following to write the current time to a csv.

index=_internal | head 1 | eval lastAlerted=now() | fields - _* | fields lastAlerted | outputcsv lastAlerted.csv

2) Then in the 'If custom condition is met' section, you paste this weird looking search:

| stats count | appendcols [| inputcsv lastAlerted.csv | fields lastAlerted] | where count>0 AND lastAlerted<now()-3600

In english its saying "alert me if there's any results for this search AND you havent already told me about this alert in the last hour."

0 Karma

zliu
Splunk Employee
Splunk Employee

My previous question was not clear, the goal here is to limit the emailed results to just 1 line in the alert email.
The user is trying to set "maxresults = 1" in alert_actions.conf to achieve that, but it doesn't work.

0 Karma

Simeon
Splunk Employee
Splunk Employee

This question is a bit confusing and may need clarification by the person asking.

Alerting is based off of the condition you dictate in your search. You can alert based on a condition with respect to the result set (events), or alert based on a schedule. All of these will be based off of some time range that your search dictates.

If you are trying to set an alert that will only send the alert one time (ever), I do not believe this is currently possible. You might be able to do some hacking with creating a summary index or lookup table, but this would depend on what you are alerting on.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.