Splunk reporting

[revanthammineni]

Hi Everyone!
I have a usecase where I need to compare daily reports and set up an alert on the deltas.

Usecase is: Hosts that are reporting yesterday, but not today.

I have the search as this.

index=os source=daily reporting hosts reporting=yes   | table host, ip | dedup host.

I'm using set command , NOT and passing time parameters but I'm not getting the right result.   

1st logic:

set diff [search index=os source=daily reporting hosts reporting=yes earliest=-3d latest=-2d  | table host, ip | dedup host] [search index=os source=daily reporting hosts reporting=yes earliest=-2d latest=-1d   | table host, ip | dedup host] | table host .

2nd logic:

[search index=os source=daily reporting hosts reporting=yes earliest=-3d latest=-2d  | table host, ip | dedup host] NOT [search index=os source=daily reporting hosts reporting=yes earliest=-2d latest=-1d   | table host, ip | dedup host] | table host .

Help me if you have any suggestions to better deal with this usecase or any change to the query.

TIA.

[gcusello]

Hi @revanthammineni,

you could run a search similar to this:

your_search(index=os source=daily reporting hosts reporting=yes)
| eval day=if(strftime(_time,"%Y-%m-%d")=strftime(now(),"%Y-%m-%d"),"Today","Yesterday")
| stats dc(day) AS dc_day values(day) AS day BY host
| where dc_day=1 AND day="Yesterday"
| table host

Ciao.

Giuseppe

[revanthammineni]

Forgot to mention that the search is a summary Report. It creates a report every midnight and push to the summary index.

So, I have to set up alert based on past two days worth data. 

[gcusello]

Hi @revanthammineni,

yes, as I said: this is an approach to use in this kind of problems.

Ciao and happy Splunking.

Giuseppe

[revanthammineni]

Let’s say I need to compare aug 23rd and aug 24th search reports and alert set up on 25th..

Can you explain me how you do with your logic??

Thanks!

[gcusello]

Hi @revanthammineni,

the logic is using eval to give a value to a a variable (in my example "day") identifying that an event is in the day of 23 ("Yesterday") or 24 ("Today").

In this way you can understand if an host sent logs yesterday and today or not.

In the filter: with the dc option in stats command your can understand when logs are present in only one day (excluding in this way the hosts that sent logs in both the days) and values(day) AS day says that logs are of yesterday and not of today (to avoid the case case that an host is sending today but not yesterday).

Ciao.

Giuseppe

[revanthammineni]

Thanks for the responses. 
another question. What if I want to automate this alert set up?  Like I want to set up everyday alert on the delta  based on comparing past two day worth of data. 

[gcusello]

Hi @revanthammineni,

save the search as an alert and schedule it once a day (after midnight) using as time frame the last two days:

earliest=-2d@d latest=-d@d

Only one hint to think:

with your approach, you check the hosts that sent logs in the last two days, highlighting the missing ones, but if you have an host that didn't send logs for more than two days, you loose and don't monitor it.

I usually prefer to manage the list of hosts to monitor in a lookup, so I'm sure about the perimeter to monitor; it's more expensive because you have to manually manage this list but more sure.

Ciao.

Giuseppe