Is it possible to dedup a subset of events?

[MatMeredith]

I have a list of different events, including some events where name="exception". These exception events have stack traces associated with them. I want to remove duplicate exceptions based on their stack_trace field, but I want to keep all non-exception events because I'm going to need them later. Can I do this?

Effectively I want to do: dedup stack_trace where name="exception"

[Karthik252]

Does dedup <field_name> keepempty=true work?

[cphair]

One way would be to create a new field that contains either the stack_trace value or a value you're pretty confident would be unique in your data, and then dedup on that.

... | eval foo=if(name=="exception", stack_trace, _time . "::" . host ."::" . name) | dedup foo | ...

You might also be able to mess with transaction or stats values(), but those might compress other fields you want to kep.

[martin_mueller]

You can create a "pretty unique" field using ... | streamstats count