Knowledge Management

Summary Index has Duplicates

aohls
Contributor

I have a search I created that runs for the last 5 minutes. I scheduled this to run every 5 minutes to update a summary index I have. I am finding duplicates on my data. Is there a better way to manage the run times to make sure there are no duplicates? I am only putting a table into the summary, there are no stat commands. Is there some way to handle this better?

0 Karma
1 Solution

aohls
Contributor

I had created a field extraction not realizing that happened automatically; this caused duplicate results.

View solution in original post

0 Karma

aohls
Contributor

I had created a field extraction not realizing that happened automatically; this caused duplicate results.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi aohls,
are your duplicates only in the last seconds of the five minutes or in all the period?
if duplicates are only in the last period, you could insert in your main search a subsearch from the summary index excluding events alreadi indexed, in other words something like this:

index=my_index NOT [ search my_symmaty_index | fields _time field1 field2 field3 ]
| table _time field1 field2 field3 
| collect index=my_summary_index

If instead are in a larger period, you should analyze again you data and your search.

Bye.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...