Alerting
Highlighted

How to compare yesterday's data with today

Communicator

Now I have a cluster.

My alerts is created on the search head of cluster and my data comes from the indexes of cluster

now , I need to create an alert, the role of this alert is: compare two days (today and yesterday) data, screening out the new content

How do I write this search statement? Assuming the index is "index = test"

If you use lookup may be a lot of problems, I tried.

alt text

0 Karma
Highlighted

Re: How to compare yesterday's data with today

Builder

You can do a distinct count to see if there are 2 distinct values in 2 days for a field -

| eventstats dc(field1) as field1_count,dc(field2) as field2_count by index | WHERE field1 > 1 AND field2 > 1
0 Karma
Highlighted

Re: How to compare yesterday's data with today

Communicator

Do these two fields must be able to determine the only one data?
Does it print out the different rows of data? Then I save as alert . How should I set the trigger condition and search span?

0 Karma
Highlighted

Re: How to compare yesterday's data with today

Motivator

If I understand you correctly you want to be alerted when a field has a different value today than yesterday.

I used some of my perfmon data to simulate this sort of situation by averaging a value by host for each day and then subtracting them to create a field named "different". In my example, I used a change of +5 in the "different" column, but you use "different!=0" to see everything that was different.

alt text

index=perfmon host=server* counter="% Processor Time" earliest=@d latest=now 
| stats avg(Value) as today by host 
| appendcols 
    [ search index=perfmon host=server* counter="% Processor Time" earliest=@d-24h latest=@d 
    | stats avg(Value) as yesterday by host] 
| eval different=today-yesterday
| search different>5
Highlighted

Re: How to compare yesterday's data with today

SplunkTrust
SplunkTrust

You would need the fields, preferably unique identifiers that you want to compare in that index. Also, it's better to narrow it down to specific sourcetype(s) which has similar data. Assuming that in index=test, there are two fields fieldA kind of primary key and fieldB which contains the data that may change, so something like this would give you events from today which have different value (fieldB) then yesterday.

index=test ...other filters if there..  earliest=-1d@d latest=now
| eval Day=if(_time<relative_time(now(),"@d"),"Yesterday","Today")
| chart values(fieldB) over fieldA by Day | where Yesterday!=Today

View solution in original post

Highlighted

Re: How to compare yesterday's data with today

Communicator

hello~I want the "diff" field to show different results.

0 Karma
Highlighted

Re: How to compare yesterday's data with today

SplunkTrust
SplunkTrust

Just add | eval Diff=Today-Yesterday to create a field showing difference in the result of two days.

0 Karma
Highlighted

Re: How to compare yesterday's data with today

Explorer

@somesoni2 is there a way to reverse the order of bars so that Yesterday appear before Today? Thank you

0 Karma
Highlighted

Re: How to compare yesterday's data with today

SplunkTrust
SplunkTrust

Just add following table statement at the end for ordering.

| table fieldA Yesterday Today
0 Karma
Highlighted

Re: How to compare yesterday's data with today

Explorer

I like that better than using | rename Before as Yesterday Now as Today
Upvoted for your help. Thank you!

0 Karma