Indexing Semi-Unified Logs

shayhibah — Mon, 23 Apr 2018 09:19:48 GMT

Hi,

In case I have data source that sends logs to my Splunk server and some logs can be sent more than once since the data source uses semi-unified logs (original log fields + all its updates).

For example:
time: 10:00 am ---- log: key=0x0001; aa=1; bb=2;
time:10:01 am --- log: key=0x0001; aa=2; bb=2;

Is there any option to delete the old log when identifying a new log with the same key?

Thanks

Re: Indexing Semi-Unified Logs

FrankVl — Mon, 23 Apr 2018 13:41:48 GMT

That's not really the idea behind ingesting data in Splunk, at least not for working with normal indexed data.

Sounds like a summary index (or lookup) in which you keep the latest entry for each key might be a direction to look into.

topic Re: Indexing Semi-Unified Logs in Splunk Dev

Indexing Semi-Unified Logs

Re: Indexing Semi-Unified Logs