All Apps and Add-ons

Can Splunk remove events before data input?

tamduong16
Contributor

I make Splunk monitors my directory and in this directory, I have lots of files. I do realize that every time I erase all content in a particular file and then rewrite it with new information, Splunk indexes new information in but I can't figure out a way to make Splunk delete old event. Is there a way for me to automatically set Splunk to delete all events in that index before it indexes new data in? And most importantly, could I do this without stopping splunk? Thanks

0 Karma
1 Solution

xpac
SplunkTrust
SplunkTrust

Hey, this isn't really possible. Splunk has no real data deletion feature, besides the | delete command, that a) only makes data invisible, but does not really delete it, and b) has to be called manually with the data that is to be "deleted".

Other than that, there is only the retention time, that can be configured per index, although it's not super precise in what it deletes (you might have older events still available). But neither this nor the delete command can be initiated by a data input.

Alternatively, if you can put your data in CSV format, you could put it in a lookup file instead if indexing it - that way you would only have the current state of the file available as data in Splunk.

View solution in original post

xpac
SplunkTrust
SplunkTrust

Hey, this isn't really possible. Splunk has no real data deletion feature, besides the | delete command, that a) only makes data invisible, but does not really delete it, and b) has to be called manually with the data that is to be "deleted".

Other than that, there is only the retention time, that can be configured per index, although it's not super precise in what it deletes (you might have older events still available). But neither this nor the delete command can be initiated by a data input.

Alternatively, if you can put your data in CSV format, you could put it in a lookup file instead if indexing it - that way you would only have the current state of the file available as data in Splunk.

View solution in original post

tamduong16
Contributor

This's a great tip. thank you

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!