How do I clean metadata after deleting the eventdata for a particular source

[sanurd]

Hello,

I indexed data using files and directory monitor to index multiple files in a folder.
I later deleted the data from the source using this search.

source="C:\splunkproject\perf\36257\*" host="AUSD1C" sourcetype="perf" | delete

I expected this to not only delete the events but also the metadata. Looks like the metadata for this source is still present.

When I run the following search, I see the event count is 0 but the metadata for the source is still present.

| metadata type=sources | where (source LIKE "%36257%")

firstTime lastTime recentTime source totalCount type
2147483647 0 1433208419 C:\splunkproject\perf\36257\A_perf_mon.log 0 sources
2147483647 0 1433208419 C:\splunkproject\perf\36257\B_perf_mon.log 0 sources

Any ideas on how to clean up the meta data.

I looked at 'splunk help clean' which doesn't have an option to clean metadata and the following post,

http://answers.splunk.com/answers/227046/how-reliable-is-the-metadata-command-and-how-can-i.html?utm...

which seems to indicate that the metadata won't be cleaned up using | delete search command.
Is there a way besides deleting the whole index ( which I don't want to do as I have other sources that I want to use )

thank you in advance.

[MuS · Accepted solution]

Hi sanurd,

Looks like the metadata for this source is still present. Yes it is, because the delete command does NOT delete events. The events are no longer searchable, but are still in the index - see docs http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/Delete

If you want to remove events and its metadata you have to use clean - but be aware this can only be done on index level.
Another useful docs link includes the hint btw http://docs.splunk.com/Documentation/Splunk/6.2.3/Indexer/RemovedatafromSplunk#Delete_events_from_su...

The delete operator does not update the metadata of the events, so any metadata searches will still include the events although they are not searchable. The main All indexed data dashboard will still show event counts for the deleted sources, hosts, or sourcetypes.

Hope that helps ...

cheers, MuS

[jaredlaney]

The delete command has never worked predictably and always causes the metadata and actual data to get out of sync. I wish Splunk would just deprecate the command because it doesn't work.

[sanurd]

Thanks MuS for your response. I was hoping to find a way to delete events and metadata for a particular source without deleting the whole index. Looks like it is not possible , Thanks anyways.