Splunk Search

Report Dedup

TucoRameriz
Explorer

Is there a way to eliminate duplicates by reports? Specifically what I'm looking to do is run a report every 24hrs for X range | stat count by shost. I don't want the report to show any hosts that showed up on the previous report. I know how to eliminate duplicates from a single report but don't know if it's possible to "dedup" on a previously run report.

Thanks

Tags (1)
0 Karma

kristian_kolb
Ultra Champion

martin_muller is essentially right, the solution below might be slightly more efficient, as the stats is performed on only the smaller set of data.

sourcetype=xxx index=yyy earliest=@d NOT [sourcetype=xxx index=yyy earliest=-1d@d latest=@d | dedup shost | fields + shost] | stats c by shost

The subsearch will execute first and return the distinct set of shost for the previous day, so that the outer search will effectively be;

sourcetype=xxx index=yyy earliest=@d NOT ((shost=host1) OR (shost=host2) OR (shost=host3)) | stats c by shost

Hope this helps,

Kristian

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You could do something like this:

your  report with stat count for the current day | search NOT [query for shost values in the previous day]

That will remove rows if the shost value appeared in the previous day... once translated into proper splunk commands of course.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...