All Apps and Add-ons

Can Splunk remove events before data input?

tamduong16
Contributor

I make Splunk monitors my directory and in this directory, I have lots of files. I do realize that every time I erase all content in a particular file and then rewrite it with new information, Splunk indexes new information in but I can't figure out a way to make Splunk delete old event. Is there a way for me to automatically set Splunk to delete all events in that index before it indexes new data in? And most importantly, could I do this without stopping splunk? Thanks

0 Karma
1 Solution

xpac
SplunkTrust
SplunkTrust

Hey, this isn't really possible. Splunk has no real data deletion feature, besides the | delete command, that a) only makes data invisible, but does not really delete it, and b) has to be called manually with the data that is to be "deleted".

Other than that, there is only the retention time, that can be configured per index, although it's not super precise in what it deletes (you might have older events still available). But neither this nor the delete command can be initiated by a data input.

Alternatively, if you can put your data in CSV format, you could put it in a lookup file instead if indexing it - that way you would only have the current state of the file available as data in Splunk.

View solution in original post

xpac
SplunkTrust
SplunkTrust

Hey, this isn't really possible. Splunk has no real data deletion feature, besides the | delete command, that a) only makes data invisible, but does not really delete it, and b) has to be called manually with the data that is to be "deleted".

Other than that, there is only the retention time, that can be configured per index, although it's not super precise in what it deletes (you might have older events still available). But neither this nor the delete command can be initiated by a data input.

Alternatively, if you can put your data in CSV format, you could put it in a lookup file instead if indexing it - that way you would only have the current state of the file available as data in Splunk.

tamduong16
Contributor

This's a great tip. thank you

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...