Splunk Search

Identical searches with different results because of dedup command

Path Finder

Hi! I have two identical searches running on the same search head but with different time frames. What confuses me is that where the searches overlap in time, the results are different from one to the other, which doesn't make much sense to me. The two searches are:

index=XXXXXXXXXXXX sourcetype=XXXXXXXXXXX earliest=0 latest=@h | dedup src_ip sortby +_time | table src_ip,_time

and

index=XXXXXXXXXXXX sourcetype=XXXXXXXXXXX  earliest=-1h@h latest=@h | dedup src_ip sortby +_time | table src_ip,_time

As you can see the searches are identical except for the time frames. When I run the second search it results in MORE events over the last hour of the search than the first search over the same last hour. The searches are run at the same time. Any ideas why this happens?

Tags (2)
0 Karma
1 Solution

Path Finder

After a few hours of inspection I actually figured out what was wrong. It's the dedup command. Because the command is used on a longer timespan in the first search than in the second it is only natural that some src_ip's might be found in the first search that are not present in the second. Because of this fewer src_ip's are deduped in the second search, resulting in more events being output!

View solution in original post

0 Karma

Path Finder

After a few hours of inspection I actually figured out what was wrong. It's the dedup command. Because the command is used on a longer timespan in the first search than in the second it is only natural that some src_ip's might be found in the first search that are not present in the second. Because of this fewer src_ip's are deduped in the second search, resulting in more events being output!

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

@tsomod, If your problem is resolved, please accept an answer (you can accept your own answer) to help future readers.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Legend

Hi tsomod,
probably you're receiving logs in the meanwhile you're searching.
try to execute searches using past and fixed time frames (e.g. -1h@h @h and -2h@h -1h@h), in other words, don't use now as latest.
Bye.
Giuseppe

0 Karma

Path Finder

After a few hours of inspection I actually figured out what was wrong. It's the dedup command. Because the command is used on a longer timespan in the first search than in the second it is only natural that some src_ip 's might be found in the first search that are not present in the second. Because of this fewer src_ip's are deduped in the second search, resulting in more events being output! Thank you, however, for your response!

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!