Splunk Search

How to find differences between two searches with "set diff" command?

evang_26
Communicator

Hi,

It's been more than a week that I am trying to display the difference between two search results in one field using the "| set diff" command diff. However, it seems to be impossible and very difficult. Below is my code:

| set diff [search sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d | eval day="Yesterday"  | timechart count(dest_ip) by dest_ip] [search sourcetype=nessus source=*Host_Enumeration* earliest=-1d@d latest=now| eval day="Today" |timechart count(dest_ip) by dest_ip]

Now, the output of that should be only one dst IP address, but what I get as a result is a big timechart with all the dst IPs. It seems like the "| set diff" command never applied to it? I don't know.

Can you help me please?

Regards,
Evang

Tags (3)
1 Solution

somesoni2
Revered Legend

Try something like this

To get list of dest_ip which are not common between Yesterday and Today.

| set diff [search sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d | stats count by dest_ip |table dest_ip] [search sourcetype=nessus source=*Host_Enumeration* earliest=-1d@d latest=now | stats count by dest_ip |table dest_ip]

If you need to know list of dest_ip present Yesterday but not Today.

sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d NOT [search sourcetype=nessus source=*Host_Enumeration*  earliest=-1d@d latest=now | stats count by dest_ip | dest_ip]| stats count by dest_ip | dest_ip

If you need to know list of dest_ip present Today but not Yesterday.

sourcetype=nessus source=*Host_Enumeration* earliest=-1d@d latest=now NOT [search sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d | stats count by dest_ip | dest_ip]| stats count by dest_ip | dest_ip

View solution in original post

somesoni2
Revered Legend

Try something like this

To get list of dest_ip which are not common between Yesterday and Today.

| set diff [search sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d | stats count by dest_ip |table dest_ip] [search sourcetype=nessus source=*Host_Enumeration* earliest=-1d@d latest=now | stats count by dest_ip |table dest_ip]

If you need to know list of dest_ip present Yesterday but not Today.

sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d NOT [search sourcetype=nessus source=*Host_Enumeration*  earliest=-1d@d latest=now | stats count by dest_ip | dest_ip]| stats count by dest_ip | dest_ip

If you need to know list of dest_ip present Today but not Yesterday.

sourcetype=nessus source=*Host_Enumeration* earliest=-1d@d latest=now NOT [search sourcetype=nessus source=*Host_Enumeration* earliest=-3d@d latest=-2d@d | stats count by dest_ip | dest_ip]| stats count by dest_ip | dest_ip

View solution in original post

ephrem3232
Explorer

Thank you so much. Took me a whole day to figure it out, but finally did it. Thank you again.

0 Karma

guru89044
Explorer

This is not working for me.

index="x" AND host= y AND "java.exception" AND earliest=-1d@d latest=now
NOT [search (index=x) AND (host=y AND "java.exception" AND earliest=-3d@d latest=-2d@d
| rex "(?java?.[.\w]+Exception)" | stats count by Exception | sort by count ] | rex "(?java?.[.\w]+Exception)" | stats count by Exception | sort by count

its not listing today' exceptions. Its listing all the exceptions from today and yesterday and ignoring common ones

0 Karma

smahone11
Engager

This example also just returns all the results from the first subsearch, even though both searches return the exact same results

0 Karma

meganarellano
Explorer

Thank you! After beating my head against a wall for a week I finally found this Answer. I removed the "stats count by" as the results were the same either way but otherwise, it worked like a charm as is.

0 Karma

evang_26
Communicator

Hi somesoni2,

Thank you so much! That worked!

Regards,
Evang

somesoni2
Revered Legend

That's my bad, I missed 'table' command there. Try the updated answer.

sarmavempati
New Member

Can you suggest how to display the results side by side, rather than one after another?
Output:
A B


1
2
3
4
5

===================

I would rather like it be:

1 4
2 5
3
etc.

Thanks

0 Karma

sarmavempati
New Member

Formatting lost!

...A.......................B...
...1........................x
...2........................x
...3.........................x
.............................4

.............................5

Desired:
.....A..............B
.....1..............4
......2.............5
......3.................
etc.

0 Karma

evang_26
Communicator

Hi somesoni2,

Thank you for your response. I would go with the first option, that is find which IPs are not in common.

However, this doesn't work prompting that "dest" command doesn't exist.

I tried to add the "search" command before dest_ip on each end. That time I saw no error coming, but no results as well.

Last, I tried removing all the last part, making it end at stats count by dest_ip. I got nothing again, though I know that there is a difference.

Hmm..

Regards,
Evangelos

0 Karma

Ayn
Legend

The output will not be what you want, because you're comparing two sets that will be completely different.

The first set will have a number of values for _time that correspond to the time periods the first search covers, which is from 3 days ago up until 2 days ago. The second set on the other hand will have times that include the last day up until now. So set diff will look at these sets, compare them and see that these are different on every line.

I'm not sure I follow what you want to do with timechart if all you want is two sets of IP's (also not sure what the eval command before that is for). If you tell us more about what you're trying to achieve it will be easier to help you. set diff would be one option, but this sounds like something that should be doable just using subsearches as well.

0 Karma

evang_26
Communicator

Hi Ayn,

Thank you for your answer!

You correctly understanded that I want to compare just two sets of IPs, nothing else.

No specific reason why I used timechart in the end. About eval, nothing special again, I saw it in an example with set diff.

I played also with the following (if that is called subsearch) without any results.

sourcetype=nessus source=Host_Enumeration earliest=-3d@d latest=-2d@d NOT [ search sourcetype=nessus source=Host_Enumeration earliest=-1d@d latest=now] | timechart count(dest_ip) by dest_ip]

Any ideas, just for the IP set comparison?

Regards,
Evangelos

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.