Splunk Search

Find duplicate events for a pattern that occurred in the same timestamp

gbehl
New Member

My requirement is to find duplicate events for a pattern that occurred in the same 'second' of timestamp after stripping the millisecond value.

queries that I tried but didn't give me 100% success:

search_pattern | timechart span=1s count | where count >1

search_pattern | timechart span=1s count | where count >1 | table _time, _raw

Not sure if 'eventcount summarize=false' or 'eventstats' would be of any help here.

P.S. I've recently started on splunk hence my knowledge is limited but I can work with pointers and do hit n trial approach.

Any pointers are appreciated.

Tags (1)
0 Karma

woodcock
Esteemed Legend

Like this:

search_pattern | bin span=1s _time | eventstats count AS duplicate_count BY _raw | search duplicate_count > 1
0 Karma

somesoni2
Revered Legend

Give this a try (would be very resource intensive so run for small time ranges)

search_pattern | bucket span=1s _time | stats count first(_raw) as raw by _time | where count>1 | rename raw as _raw
0 Karma

gbehl
New Member

It works fine except the fact that it picks the first row and ignores the other rows. After running this query, I realized that there are some exceptions in data where I would need all the _raw rows which I can get by doing an extra click. I can live with that for now.

Thanks a lot!

0 Karma

somesoni2
Revered Legend

This query is written to collect first row (as we assume they're all same, may be different millisecond). If you want all _raw data in the result, simply replace first(_raw) with values(_raw) (to get unique raw events) OR list(_raw) to get all raw events.

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...