My requirement is to find duplicate events for a pattern that occurred in the same 'second' of timestamp after stripping the millisecond value.
queries that I tried but didn't give me 100% success:
search_pattern | timechart span=1s count | where count >1
search_pattern | timechart span=1s count | where count >1 | table _time, _raw
Not sure if 'eventcount summarize=false' or 'eventstats' would be of any help here.
P.S. I've recently started on splunk hence my knowledge is limited but I can work with pointers and do hit n trial approach.
Any pointers are appreciated.
Like this:
search_pattern | bin span=1s _time | eventstats count AS duplicate_count BY _raw | search duplicate_count > 1
Give this a try (would be very resource intensive so run for small time ranges)
search_pattern | bucket span=1s _time | stats count first(_raw) as raw by _time | where count>1 | rename raw as _raw
It works fine except the fact that it picks the first row and ignores the other rows. After running this query, I realized that there are some exceptions in data where I would need all the _raw rows which I can get by doing an extra click. I can live with that for now.
Thanks a lot!
This query is written to collect first row (as we assume they're all same, may be different millisecond). If you want all _raw data in the result, simply replace first(_raw)
with values(_raw)
(to get unique raw events) OR list(_raw)
to get all raw events.