I cannot delete the events in splunk, i did append this search with delete command..I'm looking to delete the events which have "checkout/infuse.jspmethod=KRA&servervi=" this words....i do have can_delete and delete_by_keyword role to my access...
index=* sourcetype=webserver_logs source="/opt/ihs/access/*/log/access*log" "*checkout/infuse.jspmethod=KRA&servervi=*" | delete
If the job is timing out it is because you have a huge number of events being returned. When you run the search, click on the "jobs" menu and select "Send job to background" and give it an email address to send you an email when it is done. This will keep the job from timing out.
This is a different question. Click on the gear button on the upper right corner of your comment, and select "convert to question" in order to convert your answer to a question.
The solution is very simple and I'll be happy to help you once you convert this to your own question.
First your search command is fine.
Setting can_delete will allow you to delete. (Make sure you remove can_delete when you are done.)
Anything is possible with permissions but it is likely good if you are indexing data into the index
We do not have write/read permissions in our roles, splunkd writes, you read, unless you set can_delete. The roles can restrict the index you are allowed to use but you said your search returns events so you are good there.
Are your indexers clustered?
See this section in the link below: "The delete operation and indexer clusters"
As already mentioned, check splunkd.log for errors
Is/are the index(Es) you're deleting from owned by the splunkd user account? Check the filesystem permissions on the indexes to verify if so.
Also check the search.log by running your delete search and then click on inspect job, then click on search.log. Look in that log for errors, warnings, etc
I don't think delete would delete events from indexers, it only make events non searchable by users. correct me if i'm wrong
when i run the query it's just sitting there for a while and says the search job is expired. It's happening even when i run the search for 1hr or 1day.