Unable to delete search results

[sylim_splunk]

We are using 8.2.3 with SHC and multisite indexer clustering.

We have some mismatch on key business data and we need to delete and reload some data from the summary index only for a few days.

The below search returns data which is wrong and to be deleted.

Search:  index=INDEXNAME sourcetype=stash source=SOURCENAME datasource="DATASOURCENAME" host=HOSTNAME

But when I add "| delete" for the search above.  It deletes 0 events and no error.

This worked a few months ago but not today so I reckon there's no config issues, like capability and deleteIndexesAllowed has been configured for the index already.

[sylim_splunk · Accepted solution]

There is a known issue where Indexer cluster with multi-sites for the versions prior to 8.1.8 and 8.2.4. The symptoms are exactly the same as above description - the search's working fine and finds the events in question. But with "| delete" it fails to find events that you want to delete due to no primary buckets that can be searched by delete command.

- Fixed version: 8.1.8+ and 8.2.4+ ,

- Affected: versions in 8.1.0~8.1.7 or 8.2.0 ~ 8.2.3

- Workaround:  This may have some performance impact on SHC so you may want to use it when you hit the issue and revert it back.

@cm, add the below and restart the CM.

[clustering] in server.conf

assign_primaries_to_all_sites = true