Is it expected that the delete command will occasionally hang?

[dmr195]

Last year somebody found a case of delete hanging indefinitely which could be cured by deleting .lock files from the index storage area. Somebody else answered that this was caused by bugs in Splunk: SPL-84276 and SPL-81481.

I too have seen the issue with delete hanging, and it is indeed curable by deleting .lock files as described in the other answer. I saw it with Splunk 6.2.1 and upgraded to 6.2.4 and haven't seen it since, but one of my colleagues just saw it today with Splunk 6.2.3.

I have a few questions (probably only answerable by Splunk insiders) about what the expected status of the delete command is. The title of bugs SPL-84276 and SPL-81481 was "If a search process is waiting to acquire a lock then it should print lock file information in search.log". To me that doesn't sound like the fix for these bugs actually solved the potential deadlock problem, just that affected searches should detect the deadlock and log it. Is that correct? If so, does anyone know the exact circumstances under which delete hangs? Is this considered a bug that will be fixed one day? Or is it already expected to be fixed, and if so, in which version of Splunk?

[lagnone_splunk · Accepted solution]

Hi dmr195,

SPL-84276 and the original SPL-81481 do not address the root cause of a | delete search changing, they only provide a means by which to identify why a search is in hung state.

The search.log of the search with the delete command will print that the search is blocked because of "xxxxxx-xxxxx.tsidx.lock" file. This can be used by support/the user to identify a stale lock file and delete it.

From what I can tell, the potential chance of this issue occurring is not strictly limited to searches with | delete.