topic Throttle Alerts for a table of results until end of the current day in Splunk Search

Throttle Alerts for a table of results until end of the current day

njohnson7 — Wed, 07 Aug 2019 19:17:41 GMT

I am trying to setup an alert which will run every hour and considers the data from the start of current day(earliest=@d) and send the summary of table results via email. As long as the first set of results are produced I would like to throttle it until end of the current day.

When I select the trigger option ONCE , i have the values like hours, days etc for which if select 24 or 1 respectively , I am afraid my alert will trigger again only after 24 hours or a day from the time it was triggered last.

When I evaluate a Date field and try to throttle it using the PER RESULT option and using Date as a suppressing field, the alert email only has the first line & the Grand Total line from my table summary - since my table of results are lost , I cant use this method.

Could you please advise a way to keep my table result structure while letting me to throttle until end of the current day.

Desired Result:

      SYSTEM      Qname        FAIL         TRYAGAIN  Grand Total
      sys1         ABC*        300            25        325
      sys1         DEF*        210            55        265
      sys1         GHI*        470            25        495
      sys1         JK*         250            25        275
               Grand Total     1230           130       1360

Query :

index=xyz sourcetype="abc" SYSTEM="sys1" 
| dedup client NAME STATUS 
| search STATUS=FAIL OR STATUS=TRYAGAIN 
| eval "Qname"=case(NAME like "ABC%" , "ABC*", NAME like "DEF%" ,"DEF*", NAME like "GHI%" , "GHI*" , NAME like "JK%","JK*", 1=1,"Others") 
| search "Qname"!=Others 
| eventstats count as Grandtotal 
| where Grandtotal >50 
| eval Date=strftime(_time, "%d/%m/%Y") 
| stats count(eval(STATUS="FAIL")) as "FAIL" , count(eval(STATUS="TRYAGAIN")) as "TRYAGAIN" values(Date) as Date by SYSTEM, "Qname" 
| table Date SYSTEM "Qname" FAIL TRYAGAIN
| addtotals fieldname="Grand Total" col=t row=t labelfield="Qname" label="Grand Total"

Re: Throttle Alerts for a table of results until end of the current day

jacobpevans — Wed, 07 Aug 2019 21:52:51 GMT

When I evaluate a Date field and try to throttle it using the PER RESULT option and using Date as a suppressing field, the alert email only has the first line & the Grand Total line from my table summary - since my table of results are lost , I cant use this method.

I'm not fully following this. Could you try doing that again but with _time to now() and values(Date) as Date by SYSTEM, "Qname" to by Date SYSTEM "Qname" e.g.:

 index=xyz sourcetype="abc" SYSTEM="sys1" 
 | dedup client NAME STATUS 
 | search STATUS=FAIL OR STATUS=TRYAGAIN 
 | eval "Qname"=case(NAME like "ABC%" , "ABC*", NAME like "DEF%" ,"DEF*", NAME like "GHI%" , "GHI*" , NAME like "JK%","JK*", 1=1,"Others") 
 | search "Qname"!=Others 
 | eventstats count as Grandtotal 
 | where Grandtotal >50 
 | eval _Date=strftime(now(), "%d/%m/%Y") 
 | stats count(eval(STATUS="FAIL")) as "FAIL" , count(eval(STATUS="TRYAGAIN")) as "TRYAGAIN" by _Date SYSTEM "Qname" 
 | table _Date SYSTEM "Qname" FAIL TRYAGAIN
 | addtotals fieldname="Grand Total" col=t row=t labelfield="Qname" label="Grand Total"

Re: Throttle Alerts for a table of results until end of the current day

njohnson7 — Thu, 08 Aug 2019 07:05:56 GMT

@jacobevans thankyou but it is more or less similar to what I already tried. The results appears in the below format.

Date             SYSTEM      Qname        FAIL         TRYAGAIN  Grand Total
08/08/2019       sys1         ABC*        300            25        325
08/08/2019       sys1         DEF*        210            55        265
08/08/2019       sys1         GHI*        470            25        495
08/08/2019       sys1         JK*         250            25        275
            Grand Total     1230           130       1360

Now when I try to trigger the alerts using the "FOR EACH RESULT" option and giving Date as the field value for suppressing the alerts.

I only get the first line and the last line from my table of results since the date value is same:

 Date             SYSTEM      Qname        FAIL         TRYAGAIN  Grand Total
 08/08/2019       sys1         ABC*        300            25        325
                           Grand Total     1230           130       1360

I would like to send my entire table contents in the alerts in the format I have shown in the example , however I would like to suppress it only until end of the day and start over again as soon as the next day starts.