topic Why are the counts inconsistent for metadata under Data Summary after using Delete? in Splunk Search

Why are the counts inconsistent for metadata under Data Summary after using Delete?

jakewalter — Thu, 13 Nov 2014 15:49:14 GMT

After running the delete command to remove some incorrectly indexed data, the data is indeed gone from the index, but the Data Summary window in the Search app will sometimes show a count for the deleted data. This doesn't happen every time, and I can't speak to any differences between syntax when data metadata is fully deleted, versus when the metadata leaves an active count.

Is there something in Splunk that needs to be reset?

Re: Why are the counts inconsistent for metadata under Data Summary after using Delete?

MuS — Thu, 13 Nov 2014 16:44:32 GMT

Hi,

The delete command does not delete nor remove events from the index, they are no longer searchable but still in the index. See the docs for more details http://docs.splunk.com/Documentation/Splunk/6.2.0/SearchReference/Delete#Description

Cheers, MuS

Re: Why are the counts inconsistent for metadata under Data Summary after using Delete?

jakewalter — Thu, 13 Nov 2014 16:55:56 GMT

That's all well and good, but to focus on the main point of my question, what causes the metadata displayed in the Data Summary to sometimes be affected after delete (e.g. sourcetypes or hosts are removed completely), but other times partial counts for the "deleted" data remain? I know that this was a bug in older versions of Splunk, but I do not see references to this in the latest.

Re: Why are the counts inconsistent for metadata under Data Summary after using Delete?

lib_systems — Thu, 21 Apr 2016 02:02:47 GMT

I second jakewalter's follow up question as I'm experiencing this same issue on a recent install of Splunk Enterprise 6.4.0. Was a bug re-introduced?

Re: Why are the counts inconsistent for metadata under Data Summary after using Delete?

somesoni2 — Thu, 21 Apr 2016 06:24:35 GMT

In the Splunk documentation for deleting data using delete command, it has mentioned that delete command doesn't update the metadata. That's the reason you would see counts in Data summary for deleted events as well. However, those deleted event's metadata will get cleared once they go past their retention period.

http://docs.splunk.com/Documentation/Splunk/6.4.0/Indexer/RemovedatafromSplunk
he delete operator does not update the metadata of the events, so any metadata searches will still include the events although they are not searchable. The main All indexed data dashboard will still show event counts for the deleted sources, hosts, or sourcetypes.

Re: Why are the counts inconsistent for metadata under Data Summary after using Delete?

MuS — Thu, 21 Apr 2016 07:37:30 GMT

Good spotting @somesoni2 😉

Re: Why are the counts inconsistent for metadata under Data Summary after using Delete?

ckroger — Wed, 15 Jun 2016 19:57:19 GMT

I third the follow up question. Experiencing the same problem. Once every few days, a deletion will not be reflected in the metadata - even after 24 hours. (Splunk Ent. v6.3.4)