Splunk Enterprise

Eval command questions- How many duplicated events do we have?

sqauyyu123
New Member

How many duplicated events we have? Percent of duplicated events?

Difference between duplicated and unique events.?

Labels (1)
0 Karma

johnhuang
Motivator

It depends on how you define or considered to be a duplicate event. Assuming you're talking about the _raw event data you can try this query:

 

<base_search>
| eval event_hash=MD5(nt_host)
| stats count AS event_ct by event_hash
| appendpipe [| stats sum(event_ct) AS total_ct]
| appendpipe [| stats count AS unique_ct]
| where event_ct>1 OR ISNULL(event_ct)
| appendpipe [| where ISNOTNULL(event_ct) | stats count AS events_with_dupe_ct]
| stats max(*_ct) AS *_ct
| eval redundant_events_ct=total_ct-unique_ct
| foreach *_ct [| eval <<MATCHSTR>>_per=ROUND((<<FIELD>>/total_ct)*100, 2)." %"]
| table unique_ct unique_per events_with_dupe_ct events_with_dupe_per redundant_events_ct redundant_events_per total_ct
  •  

 

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...