Splunk Search

Is it possible to dedup a subset of events?

MatMeredith
Path Finder

I have a list of different events, including some events where name="exception". These exception events have stack traces associated with them. I want to remove duplicate exceptions based on their stack_trace field, but I want to keep all non-exception events because I'm going to need them later. Can I do this?

Effectively I want to do: dedup stack_trace where name="exception"

Tags (1)
0 Karma

Karthik252
Observer

Does dedup <field_name> keepempty=true work?

0 Karma

cphair
Builder

One way would be to create a new field that contains either the stack_trace value or a value you're pretty confident would be unique in your data, and then dedup on that.


... | eval foo=if(name=="exception", stack_trace, _time . "::" . host ."::" . name) | dedup foo | ...

You might also be able to mess with transaction or stats values(), but those might compress other fields you want to kep.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You can create a "pretty unique" field using ... | streamstats count

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...