How to run a diff search with a Head 2 command across multiple systems?

[aferone]

I have developed a search, with help years ago, that will show differences in a netstat command using "diff" and "head 2".

index=foo host=bar sourcetype=netstat 
| head 2 
| diff
| search NOT "Results are the Same"

The netstat runs every hour with a scripted input, and the search runs hourly to see if anything has changed. If it has, an alert fires.

As you can see, the search specifies a host. However, what if I wanted to run this search across many hosts? Would I have to create a separate search for each host? Or is there some Splunk magic I can utilize?

Thanks for your time!

[somesoni2]

The diff command can only compare 2 result/row, so having multiple host entries would not work. If you don't need "diff command type output" and just looking to alert when current netstat output of a host is different then previous entry, you can do something like this.

index=foo sourcetype=netstat 
| table host _raw | dedup 2 host
| streamstats count as sno by host
| chart values(_raw) over host by sno
| where '1'!='2'

The dedup command will just list two entries for a host, most recent and 2nd recent. The streamstats command just give a serial number to them which'll be 1 and 2 since there will be only two entries after dedup. The chart command will give a output with field host, 1 (which will have most recent event's raw data) and 2 (which will have 2nd recent event's raw data). The where clause just compare both.

[aferone]

This looks very promising. I am pushing our netstat config to more boxes to test this. Where does "sno" come into play in the search? Thanks again!!

[aferone]

The search is definitely working, and thank you!

But for some reason, when I set it up as an alert, I can't get it to send an email? I am setting it to "events greater to zero", just like every other alert we've configured.

Am I missing something?

Thanks!

[aferone]

Sorry for the delay, and thank you for posting! . I will be trying this soon. Thanks again!

[pradeepkumarg]

Can you not do dedup host?

 index=foo host=* sourcetype=netstat 

 | dedup 2  host

[aferone]

Can you explain this a little further on what this would be doing? Thanks!

[pradeepkumarg]

so | head 2 will give you the latest 2 entries for the host you specified.

by making host=* and adding | dedup 2 host, you are retaining the latest 2 entries for every host