topic Re: Monitor for delete attempts in Monitoring Splunk

Monitor for delete attempts

BB2 — Thu, 29 May 2025 12:30:12 GMT

Hello,

Our company has gone through an audit and one of the auditors has asked us to monitor attempts to delete records in Splunk. I did some research and found the search item below which would do the trick. The issue is if I setup an alert with this, the alert is triggered because the previous search for this alert is saved and we get alerted for that search because the word delete is in that search.

index=_audit action=search | regex search="\\|(\\s|\\n|\\r|([\\s\\S]*))*delete"

Is there a way to ignore this search string when doing a search? Or has anybody been able to setup an alert for attempts to delete records?

We only have 4 admins with the can_delete role but the auditors want to be sure if an admin tries to delete records, there will be an alert.

Re: Monitor for delete attempts

livehybrid — Thu, 29 May 2025 12:58:37 GMT

Hi @BB2

You could use the the following to search for a failed attempt or success delete:

(index=_internal "You do not have the capability to delete") OR (index=_audit action=delete_by_keyword info=granted)

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Monitor for delete attempts

BB2 — Thu, 29 May 2025 14:44:36 GMT

No, that didn't help. I tested deleting data on our test server but your search did return any results. I get the error below when trying to delete. I updated your search with what is in bold but it also did not return any results.

Error in 'delete' command: You have insufficient privileges to delete events.

I found a way to search for failed deletes by adding the info=failed. That would take care of failed attempts to delete data but would not be helpful if an admin performed a delete in a search.

index=_audit action=search info=failed | regex search="\\|(\\s|\\n|\\r|([\\s\\S]*))*delete"

Re: Monitor for delete attempts

livehybrid — Thu, 29 May 2025 15:12:59 GMT

Apologies @BB2

How about just

index=_audit action=delete_by_keyword

You will get granted if success or denied if they didnt have permission:

I dont know if you're aware but you can set deleteIndexesAllowed for a role, for the can_delete role this is set to * which means any index but DOES NOT cover _* indexes. So the can_delete role wouldnt be able to delete _internal or _audit data.

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Monitor for delete attempts

BB2 — Thu, 29 May 2025 15:58:58 GMT

That will only tell me when the role delete_by_keyword has been assigned. It will not tell me if someone deletes data from them main index. I saw the role get assigned with that search as I added the can_delete role to the admin role.

Re: Monitor for delete attempts

livehybrid — Thu, 29 May 2025 16:35:13 GMT

Hi @BB2

That doesnt get logged when the capability is assigned, it is logged when the capability is attempted to be used.

info=denied means they werent successful running |delete

info=granted means they were successful.

Hopefully this clears it up.

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Monitor for delete attempts

livehybrid — Thu, 29 May 2025 16:47:16 GMT

These are the logs I get when doing a delete before having the capability, then adding the capability, then being able to delete with the new capability.

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Monitor for delete attempts

BB2 — Thu, 29 May 2025 16:47:48 GMT

I am curious then why I do not get any results if I search for info=denied. I had attempted to delete a record several times and got the insufficient privileges but nothing showed up in the audit.log.

index=_audit action=delete_by_keyword info=denied

Re: Monitor for delete attempts

PickleRick — Thu, 29 May 2025 18:58:49 GMT

As usual with similar "monitoring" searches - this will not find searches when delete isn't invoked directly. The obvious way to do so would be with a macro.

Re: Monitor for delete attempts

BB2 — Thu, 29 May 2025 19:37:08 GMT

Could you tell me how to set that up? Or, can you point me to a location on how to do it?

Re: Monitor for delete attempts

PickleRick — Thu, 29 May 2025 20:15:10 GMT

What do you mean by "how to set that up"?

Re: Monitor for delete attempts

BB2 — Thu, 29 May 2025 20:18:48 GMT

I understand this better now.

Using the search you provided will search for anyone who did delete a record. I can then go into Splunk and search the timeframe for who actually performed the delete.

index=_audit action=delete_by_keyword info=granted

I can setup an alert using this search for any failure attempts to delete a record.

index=_audit action=search info=failed | regex search="\\|(\\s|\\n|\\r|([\\s\\S]*))*delete"