Building for the Splunk Platform

Indexing Semi-Unified Logs

Path Finder


In case I have data source that sends logs to my Splunk server and some logs can be sent more than once since the data source uses semi-unified logs (original log fields + all its updates).

For example:
time: 10:00 am ---- log: key=0x0001; aa=1; bb=2;
time:10:01 am --- log: key=0x0001; aa=2; bb=2;

Is there any option to delete the old log when identifying a new log with the same key?


Tags (1)
0 Karma

Ultra Champion

That's not really the idea behind ingesting data in Splunk, at least not for working with normal indexed data.

Sounds like a summary index (or lookup) in which you keep the latest entry for each key might be a direction to look into.

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

Observability Newsletter Highlights | March 2023

 March 2023 | Check out the latest and greatestSplunk APM's New Tag Filter ExperienceSplunk APM has updated ...

Security Newsletter Updates | March 2023

 March 2023 | Check out the latest and greatestUnify Your Security Operations with Splunk Mission Control The ...