topic How to compare yesterday's data with today in Alerting

How to compare yesterday's data with today

xsstest — Wed, 26 Apr 2017 10:01:10 GMT

Now I have a cluster.

My alerts is created on the search head of cluster and my data comes from the indexes of cluster

now , I need to create an alert, the role of this alert is: compare two days (today and yesterday) data, screening out the new content

How do I write this search statement? Assuming the index is "index = test"

If you use lookup may be a lot of problems, I tried.

Re: How to compare yesterday's data with today

dineshraj9 — Wed, 26 Apr 2017 10:13:12 GMT

You can do a distinct count to see if there are 2 distinct values in 2 days for a field -

| eventstats dc(field1) as field1_count,dc(field2) as field2_count by index | WHERE field1 > 1 AND field2 > 1

Re: How to compare yesterday's data with today

xsstest — Wed, 26 Apr 2017 10:51:20 GMT

Do these two fields must be able to determine the only one data?
Does it print out the different rows of data? Then I save as alert . How should I set the trigger condition and search span?

Re: How to compare yesterday's data with today

lycollicott — Wed, 26 Apr 2017 13:23:30 GMT

If I understand you correctly you want to be alerted when a field has a different value today than yesterday.

I used some of my perfmon data to simulate this sort of situation by averaging a value by host for each day and then subtracting them to create a field named "different". In my example, I used a change of +5 in the "different" column, but you use "different!=0" to see everything that was different.

index=perfmon host=server* counter="% Processor Time" earliest=@d latest=now 
| stats avg(Value) as today by host 
| appendcols 
    [ search index=perfmon host=server* counter="% Processor Time" earliest=@d-24h latest=@d 
    | stats avg(Value) as yesterday by host] 
| eval different=today-yesterday
| search different>5

Re: How to compare yesterday's data with today

somesoni2 — Wed, 26 Apr 2017 16:48:14 GMT

You would need the fields, preferably unique identifiers that you want to compare in that index. Also, it's better to narrow it down to specific sourcetype(s) which has similar data. Assuming that in index=test, there are two fields fieldA kind of primary key and fieldB which contains the data that may change, so something like this would give you events from today which have different value (fieldB) then yesterday.

index=test ...other filters if there..  earliest=-1d@d latest=now
| eval Day=if(_time<relative_time(now(),"@d"),"Yesterday","Today")
| chart values(fieldB) over fieldA by Day | where Yesterday!=Today

Re: How to compare yesterday's data with today

xsstest — Tue, 09 May 2017 03:23:47 GMT

hello~I want the "diff" field to show different results.

Re: How to compare yesterday's data with today

somesoni2 — Tue, 09 May 2017 03:32:39 GMT

Just add | eval Diff=Today-Yesterday to create a field showing difference in the result of two days.

Re: How to compare yesterday's data with today

maraman_splunk — Tue, 09 May 2017 12:44:22 GMT

Hi,

have a look a timewrap command (see https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Timewrap , for old splunk version, you'll need the timewrap app https://splunkbase.splunk.com/app/1645/ )

That might do the work for you !

Re: How to compare yesterday's data with today

nickrally2009 — Mon, 16 Mar 2020 18:14:32 GMT

@somesoni2 is there a way to reverse the order of bars so that Yesterday appear before Today? Thank you

Re: How to compare yesterday's data with today

somesoni2 — Tue, 17 Mar 2020 05:36:44 GMT

Just add following table statement at the end for ordering.

| table fieldA Yesterday Today

Re: How to compare yesterday's data with today

nickrally2009 — Tue, 17 Mar 2020 19:17:05 GMT

I like that better than using | rename Before as Yesterday Now as Today
Upvoted for your help. Thank you!

Re: How to compare yesterday's data with today

rock_s — Thu, 11 Jun 2020 08:02:28 GMT

Hi @somesoni2 , I have few of errors like field name is errors, I would like to compare last 24 hours errors with last 60 days, if not matched, i need to display those errors. Please help on this. Thanks in advance.

Re: How to compare yesterday's data with today

gcusello — Thu, 18 Jun 2020 07:05:05 GMT

Hi @xsstest ,

you could use an approach like this (this is an example because I haven't your search):

index=wineventlog | eval day=if(date_mday=strftime(now(),"%d"),"today","yesterday") | stats count(eval(day="yesterday")) AS yesterday count(eval(day="today")) AS today BY EventCode

Then make all the filters you like (greater than, less than, perc of, etc...).

Ciao.

Giuseppe