topic Re: Why deleted events reappear in indexcluster? in Deployment Architecture

Why deleted events reappear in indexcluster?

marcohoffmann — Thu, 07 Nov 2019 06:21:11 GMT

Hi community,
Sometimes we have to delete events in splunk especially because some GDPR reasons.
After 5 year using ~20 standalone indexer we switched to indexcluster and now deleting events is not safe anymore.
Some deleted events reappear randomly after a few weeks.
Aren't the events not marked as deleted in the replicas? If so, how can mark all copies as deleted?

best regards
Mrco

Re: Why deleted events reappear in indexcluster?

jhornsby_splunk — Fri, 08 Nov 2019 17:08:03 GMT

Hi @marcohoffmann,

What version of Splunk is this occurring on, and is it a single- or multi- site cluster?

Cheers,

- Jo.

Re: Why deleted events reappear in indexcluster?

inawaz123 — Mon, 11 Nov 2019 03:29:40 GMT

We experienced same behaviors, i have came up with my own process in ansible to run a cli command from backend via cli and delete event in individual indexer peers rather than user cli, but that way it is very clean and i would be able to do this cleanup effectively.

Re: Why deleted events reappear in indexcluster?

marcohoffmann — Mon, 11 Nov 2019 07:07:56 GMT

Hi,
as I understand it is a multi-site cluster with 2 sites. The replicas are always forced to be on the other site. For searchhead we have no special conditions.

We run a relative old version 6.4.4.

Cheers,
Marco

Re: Why deleted events reappear in indexcluster?

jhornsby_splunk — Mon, 11 Nov 2019 12:38:11 GMT

Hi @marcohoffmann,

Ah, okay. So 6.4.4 does not have the fix for SPL-136734/SPL-100516, which addresses multiple issues with delete propagation, so upgrading may address your issue. However we are also tracking an issue internally, SPL-138846, which is specific to multisite clusters. Unfortunately there's no fix available for that issue as yet, but it is being worked on.

Cheers,

- Jo.

Re: Why deleted events reappear in indexcluster?

marcohoffmann — Wed, 13 Nov 2019 06:06:16 GMT

Thank you, we will do it asap. For some interests, where can I found in which release this issues SPL-136734/SPL-100516 were solved?

Re: Why deleted events reappear in indexcluster?

jhornsby_splunk — Wed, 13 Nov 2019 10:01:10 GMT

Hi @marcohoffmann,

For the 6.4 series, it was fixed in 6.4.7; see here: https://docs.splunk.com/Documentation/Splunk/6.4.7/ReleaseNotes/6.4.7

For later versions, it's fixed in 6.5.3, and all versions from 6.6.0 onwards.

Good luck! &:)

Cheers,

- Jo.

Re: Why deleted events reappear in indexcluster?

HansWurscht — Fri, 21 Feb 2020 17:03:43 GMT

We are using a multisite indexcluster (replicate 1 copy to a different location) and we are also having issues using 'delete' from our SHC.

Events are reappering after we issued the delete command.

Is there still a bug or is the mentioned multisite issue already fixed?
We are currently on 7.3.3.

Re: Why deleted events reappear in indexcluster?

jhornsby_splunk — Mon, 24 Feb 2020 12:08:23 GMT

Hi @HansWurscht,

The fix for SPL-138846 is still in progress.

Cheers,

- Jo.