Splunk Enterprise

Eval command questions- How many duplicated events do we have?

sqauyyu123
New Member

How many duplicated events we have? Percent of duplicated events?

Difference between duplicated and unique events.?

Labels (1)
0 Karma

johnhuang
Motivator

It depends on how you define or considered to be a duplicate event. Assuming you're talking about the _raw event data you can try this query:

 

<base_search>
| eval event_hash=MD5(nt_host)
| stats count AS event_ct by event_hash
| appendpipe [| stats sum(event_ct) AS total_ct]
| appendpipe [| stats count AS unique_ct]
| where event_ct>1 OR ISNULL(event_ct)
| appendpipe [| where ISNOTNULL(event_ct) | stats count AS events_with_dupe_ct]
| stats max(*_ct) AS *_ct
| eval redundant_events_ct=total_ct-unique_ct
| foreach *_ct [| eval <<MATCHSTR>>_per=ROUND((<<FIELD>>/total_ct)*100, 2)." %"]
| table unique_ct unique_per events_with_dupe_ct events_with_dupe_per redundant_events_ct redundant_events_per total_ct
  •  

 

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...