Remove data from Index

[efelder0]

I have indexed many months worth of data, but would like to "remove" only the first of the 3 months worth of data. However, I cannot clean out the entire index. Is this possible with the clean eventdata command?

[MuS · Accepted solution]

Hi efelder0

you can search for the data you no longer need and append

| delete 

to it. This data will then no longer searchable but still is in the index.

If this helps.

Cheers,
MuS

[jtashiro]

Can someone as Splunk explain the purpose of "delete" command, if it doesn't actually delete data from an index, but makes it un-searchable. As I understand it, "delete" operation is irreversible, the deleted data continues to consume disk space, and there is no way to free that up? Doesn't make sense to me. Am I not understanding it?

[botkindl]

It's very useful in some cases. For instance, we had an issue where logrotate was rotating syslogs and Splunk was indexing them (fixed with a blacklist entry). Users were getting totally confused by the "extra" hostnames, which were actually filenames from the rotated files -- and the log messaegs were duplicated as well. So we ran a search, piped to delete, everyone is happy.

At the same time, I don't ever have to explain (or defend) to our internal audit folks how and why we can actually delete data. No matter if we delete it or not, it's still there in the rawdata files and still can be found if needed. I think it's a good compromise of being able to remove extraneous/distracting search results, and being able to say that the data is permanent.

[aaraneta_splunk]

Hi @jtashiro,

Have you checked out the accepted answer at this link? It may be a good place to start.

However, if you are not satisfied with that explanation, I would suggest posting a new question about this topic since this post is from over 3 years ago and may not get the visibility you would like in order to help you.

[jtashiro]

I've read the accepted answer, and it doesn't satisfy my question. The question is best answered by Splunk technical team, with insight into why 'delete' was built to hide/mask data, but not actually 'delete' it and free up space. The 'delete' command is inaccurately and poorly named.

[linu1988]

Adding to that, metadata will be still be available. That can't be removed with delete..