How to find differences between two searches with "set diff" command?

[evang_26]

Hi,

It's been more than a week that I am trying to display the difference between two search results in one field using the "| set diff" command diff. However, it seems to be impossible and very difficult. Below is my code:

| set diff [search sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d | eval day="Yesterday"  | timechart count(dest_ip) by dest_ip] [search sourcetype=nessus source=*Host_Enumeration* earliest=-1d@d latest=now| eval day="Today" |timechart count(dest_ip) by dest_ip]

Now, the output of that should be only one dst IP address, but what I get as a result is a big timechart with all the dst IPs. It seems like the "| set diff" command never applied to it? I don't know.

Can you help me please?

Regards,
Evang

[somesoni2 · Accepted solution]

Try something like this

To get list of dest_ip which are not common between Yesterday and Today.

| set diff [search sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d | stats count by dest_ip |table dest_ip] [search sourcetype=nessus source=*Host_Enumeration* earliest=-1d@d latest=now | stats count by dest_ip |table dest_ip]

If you need to know list of dest_ip present Yesterday but not Today.

sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d NOT [search sourcetype=nessus source=*Host_Enumeration*  earliest=-1d@d latest=now | stats count by dest_ip | dest_ip]| stats count by dest_ip | dest_ip

If you need to know list of dest_ip present Today but not Yesterday.

sourcetype=nessus source=*Host_Enumeration* earliest=-1d@d latest=now NOT [search sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d | stats count by dest_ip | dest_ip]| stats count by dest_ip | dest_ip

[ephrem3232]

Thank you so much. Took me a whole day to figure it out, but finally did it. Thank you again.

[guru89044]

This is not working for me.

index="x" AND host= y AND "java.exception" AND earliest=-1d@d latest=now
NOT [search (index=x) AND (host=y AND "java.exception" AND earliest=-3d@d latest=-2d@d
| rex "(?java?.[.\w]+Exception)" | stats count by Exception | sort by count ] | rex "(?java?.[.\w]+Exception)" | stats count by Exception | sort by count

its not listing today' exceptions. Its listing all the exceptions from today and yesterday and ignoring common ones

[smahone11]

This example also just returns all the results from the first subsearch, even though both searches return the exact same results

[meganarellano]

Thank you! After beating my head against a wall for a week I finally found this Answer. I removed the "stats count by" as the results were the same either way but otherwise, it worked like a charm as is.

[evang_26]

Hi somesoni2,

Thank you so much! That worked!

Regards,
Evang

[somesoni2]

That's my bad, I missed 'table' command there. Try the updated answer.

[sarmavempati]

Can you suggest how to display the results side by side, rather than one after another?
Output:
A B

---

1
2
3
4
5

===================

I would rather like it be:

1 4
2 5
3
etc.

Thanks

[sarmavempati]

Formatting lost!

...A.......................B...
...1........................x
...2........................x
...3.........................x
.............................4

.............................5

Desired:
.....A..............B
.....1..............4
......2.............5
......3.................
etc.

[evang_26]

Hi somesoni2,

Thank you for your response. I would go with the first option, that is find which IPs are not in common.

However, this doesn't work prompting that "dest" command doesn't exist.

I tried to add the "search" command before dest_ip on each end. That time I saw no error coming, but no results as well.

Last, I tried removing all the last part, making it end at stats count by dest_ip. I got nothing again, though I know that there is a difference.

Hmm..

Regards,
Evangelos

[Ayn]

The output will not be what you want, because you're comparing two sets that will be completely different.

The first set will have a number of values for _time that correspond to the time periods the first search covers, which is from 3 days ago up until 2 days ago. The second set on the other hand will have times that include the last day up until now. So set diff will look at these sets, compare them and see that these are different on every line.

I'm not sure I follow what you want to do with timechart if all you want is two sets of IP's (also not sure what the eval command before that is for). If you tell us more about what you're trying to achieve it will be easier to help you. set diff would be one option, but this sounds like something that should be doable just using subsearches as well.

[evang_26]

Hi Ayn,

Thank you for your answer!

You correctly understanded that I want to compare just two sets of IPs, nothing else.

No specific reason why I used timechart in the end. About eval, nothing special again, I saw it in an example with set diff.

I played also with the following (if that is called subsearch) without any results.

sourcetype=nessus source=Host_Enumeration earliest=-3d@d latest=-2d@d NOT [ search sourcetype=nessus source=Host_Enumeration earliest=-1d@d latest=now] | timechart count(dest_ip) by dest_ip]

Any ideas, just for the IP set comparison?

Regards,
Evangelos