topic Re: About delete command error. in Getting Data In

About delete command error.

yutaka1005 — Tue, 16 Jan 2018 10:55:14 GMT

In my environment SH, indexer 1, indexer 2 exist, and distributed search is done for indexers 1 and 2 from SH.

Yesterday, since data was duplicated in indexers 1 and 2, I give can_delete role to admin user of SH, and executed delete command on SH.
However, despite all the data of indexer 1 being displayed as "deleted", all the data of indexer 2 was "errors".
Also, the following error message appeared.

["hostname of indexer 2"] You do not have the capability to delete from "index name"

However, when I executed the same command again, all the data of indexer 2 was "deleted" this time.

I thought that connection of SH between indexer 2 was being disconnected when I executed delete command and received error messages,
but there was no error that communication with the indexer 2 was interrupted.

What is this phenomenon?
It will be very helpful if someone tells me.

Re: About delete command error.

inventsekar — Wed, 17 Jan 2018 04:28:33 GMT

hope the indexers are not clustered, right.

One more thing....were you trying to delete data from real time search?

Note: You cannot run the delete command during a real-time search; you cannot delete events as they come in. If you try to use delete during a real-time search, Splunk Enterprise will display an error.

Re: About delete command error.

yutaka1005 — Wed, 17 Jan 2018 06:17:13 GMT

Thank you for comment!

First, my indexers are not clustered.

Second, I excuted delete with the specified search period not real-time search.

Re: About delete command error.

inventsekar — Wed, 17 Jan 2018 06:24:36 GMT

Ohk sure. So, the issue is that, when you ran first, you got error.. on second run, delete worked fine. Right?

Is it a repeating issue? Try to rerun delete command for some sample logs (careful, data deleted is irreversible, ..once deleted, data can't be retrieved back)

Re: About delete command error.

nickhills — Wed, 17 Jan 2018 08:39:29 GMT

Did you see any errors in the splunkd or webservice log?

Since re-running it worked correctly, I would be inclined to think it a transient error - but any logs indicating network connectivity issues would give some comfort that the issue was further down the stack than Splunk.

Re: About delete command error.

yutaka1005 — Thu, 18 Jan 2018 06:03:29 GMT

Yes, you right.

No it is first time.
I want to retry, but I can not easily take sample logs and delete it...

Re: About delete command error.

yutaka1005 — Thu, 18 Jan 2018 06:05:58 GMT

No, I looked for it but I could not find it.

I thought it is transient error too, but I can not find anything that can prove it ...

Re: About delete command error.

davpx — Thu, 18 Jan 2018 23:50:26 GMT

The reason this most likely failed on the 2nd indexer is that your authorization change may have yet to propagate to that indexer. By the time you tried again, the change made its way to indexer 2's bundle.

A bit off topic but may be related, how big is your search bundle and are you seeing messages for it taking longer than expected to replicate?

Re: About delete command error.

yutaka1005 — Fri, 19 Jan 2018 00:45:15 GMT

Thank you for answer!

The first sentence implies that propagation of information that "can_delete" was given to the "admin" user was delayed?

Excuse me
What does the second sentence mean?

Re: About delete command error.

davpx — Fri, 19 Jan 2018 01:23:19 GMT

A search bundle is basically a copy of your search head configurations which are replicated to your indexer. If you have a lot of apps or some that contain large files, it can sometimes take awhile to replicate.

Re: About delete command error.

yutaka1005 — Fri, 19 Jan 2018 04:00:42 GMT

I got it.

In my environment, there are not such many apps.
So I do not think there will be a long time to replicate.