Can Splunk remove events before data input?

[tamduong16]

I make Splunk monitors my directory and in this directory, I have lots of files. I do realize that every time I erase all content in a particular file and then rewrite it with new information, Splunk indexes new information in but I can't figure out a way to make Splunk delete old event. Is there a way for me to automatically set Splunk to delete all events in that index before it indexes new data in? And most importantly, could I do this without stopping splunk? Thanks

[xpac · Accepted solution]

Hey, this isn't really possible. Splunk has no real data deletion feature, besides the | delete command, that a) only makes data invisible, but does not really delete it, and b) has to be called manually with the data that is to be "deleted".

Other than that, there is only the retention time, that can be configured per index, although it's not super precise in what it deletes (you might have older events still available). But neither this nor the delete command can be initiated by a data input.

Alternatively, if you can put your data in CSV format, you could put it in a lookup file instead if indexing it - that way you would only have the current state of the file available as data in Splunk.

[tamduong16]

This's a great tip. thank you