Eval command questions- How many duplicated events do we have?

[sqauyyu123]

How many duplicated events we have? Percent of duplicated events?

Difference between duplicated and unique events.?

[johnhuang]

It depends on how you define or considered to be a duplicate event. Assuming you're talking about the _raw event data you can try this query:

<base_search>
| eval event_hash=MD5(nt_host)
| stats count AS event_ct by event_hash
| appendpipe [| stats sum(event_ct) AS total_ct]
| appendpipe [| stats count AS unique_ct]
| where event_ct>1 OR ISNULL(event_ct)
| appendpipe [| where ISNOTNULL(event_ct) | stats count AS events_with_dupe_ct]
| stats max(*_ct) AS *_ct
| eval redundant_events_ct=total_ct-unique_ct
| foreach *_ct [| eval <<MATCHSTR>>_per=ROUND((<<FIELD>>/total_ct)*100, 2)." %"]
| table unique_ct unique_per events_with_dupe_ct events_with_dupe_per redundant_events_ct redundant_events_per total_ct

•