Archive

Remove data from Index

Communicator

I have indexed many months worth of data, but would like to "remove" only the first of the 3 months worth of data. However, I cannot clean out the entire index. Is this possible with the clean eventdata command?

Tags (1)
0 Karma
1 Solution

SplunkTrust
SplunkTrust

Hi efelder0

you can search for the data you no longer need and append

| delete 

to it. This data will then no longer searchable but still is in the index.

If this helps.

Cheers,
MuS

View solution in original post

SplunkTrust
SplunkTrust

Hi efelder0

you can search for the data you no longer need and append

| delete 

to it. This data will then no longer searchable but still is in the index.

If this helps.

Cheers,
MuS

View solution in original post

New Member

Can someone as Splunk explain the purpose of "delete" command, if it doesn't actually delete data from an index, but makes it un-searchable. As I understand it, "delete" operation is irreversible, the deleted data continues to consume disk space, and there is no way to free that up? Doesn't make sense to me. Am I not understanding it?

0 Karma

Explorer

It's very useful in some cases. For instance, we had an issue where logrotate was rotating syslogs and Splunk was indexing them (fixed with a blacklist entry). Users were getting totally confused by the "extra" hostnames, which were actually filenames from the rotated files -- and the log messaegs were duplicated as well. So we ran a search, piped to delete, everyone is happy.

At the same time, I don't ever have to explain (or defend) to our internal audit folks how and why we can actually delete data. No matter if we delete it or not, it's still there in the rawdata files and still can be found if needed. I think it's a good compromise of being able to remove extraneous/distracting search results, and being able to say that the data is permanent.

Splunk Employee
Splunk Employee

Hi @jtashiro,

Have you checked out the accepted answer at this link? It may be a good place to start.

However, if you are not satisfied with that explanation, I would suggest posting a new question about this topic since this post is from over 3 years ago and may not get the visibility you would like in order to help you.

0 Karma

New Member

I've read the accepted answer, and it doesn't satisfy my question. The question is best answered by Splunk technical team, with insight into why 'delete' was built to hide/mask data, but not actually 'delete' it and free up space. The 'delete' command is inaccurately and poorly named.

0 Karma

Champion

Adding to that, metadata will be still be available. That can't be removed with delete..

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!