Monitoring Splunk

Monitor for delete attempts

BB2
Explorer

Hello,

Our company has gone through an audit and one of the auditors has asked us to monitor attempts to delete records in Splunk.  I did some research and found the search item below which would do the trick.  The issue is if I setup an alert with this, the alert is triggered because the previous search for this alert is saved and we get alerted for that search because the word delete is in that search.  

index=_audit action=search | regex search="\\|(\\s|\\n|\\r|([\\s\\S]*))*delete"

Is there a way to ignore this search string when doing a search?  Or has anybody been able to setup an alert for attempts to delete records?

We only have 4 admins with the can_delete role but the auditors want to be sure if an admin tries to delete records, there will be an alert.

 

Labels (1)
0 Karma
1 Solution

livehybrid
Super Champion

Apologies @BB2 

How about just 

index=_audit action=delete_by_keyword

You will get granted if success or denied if they didnt have permission:

livehybrid_0-1748531505259.png

 

I dont know if you're aware but you can set deleteIndexesAllowed for a role, for the can_delete role this is set to * which means any index but DOES NOT cover _* indexes. So the can_delete role wouldnt be able to delete _internal or _audit data.

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

View solution in original post

PickleRick
SplunkTrust
SplunkTrust

As usual with similar "monitoring" searches - this will not find searches when delete isn't invoked directly. The obvious way to do so would be with a macro.

0 Karma

BB2
Explorer

Could you tell me how to set that up?  Or, can you point me to a location on how to do it?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

What do you mean by "how to set that up"?

0 Karma

livehybrid
Super Champion

Hi @BB2 

You could use the the following to search for a failed attempt or success delete:

(index=_internal "You do not have the capability to delete") OR (index=_audit action=delete_by_keyword info=granted)

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

 

0 Karma

BB2
Explorer

No, that didn't help.  I tested deleting data on our test server but your search did return any results.  I get the error below when trying to delete. I updated your search with what is in bold but it also did not return any results.

Error in 'delete' command: You have insufficient privileges to delete events.

I found a way to search for failed deletes by adding the info=failed.  That would take care of failed attempts to delete data but would not be helpful if an admin performed a delete in a search.

index=_audit action=search info=failed | regex search="\\|(\\s|\\n|\\r|([\\s\\S]*))*delete"

0 Karma

livehybrid
Super Champion

Apologies @BB2 

How about just 

index=_audit action=delete_by_keyword

You will get granted if success or denied if they didnt have permission:

livehybrid_0-1748531505259.png

 

I dont know if you're aware but you can set deleteIndexesAllowed for a role, for the can_delete role this is set to * which means any index but DOES NOT cover _* indexes. So the can_delete role wouldnt be able to delete _internal or _audit data.

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

BB2
Explorer

That will only tell me when the role delete_by_keyword has been assigned.  It will not tell me if someone deletes data from them main index.  I saw the role get assigned with that search as I added the can_delete role to the admin role.  

 

 

0 Karma

livehybrid
Super Champion

These are the logs I get when doing a delete before having the capability, then adding the capability, then being able to delete with the new capability.

livehybrid_0-1748537231117.png

 

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

0 Karma

livehybrid
Super Champion

Hi @BB2 

That doesnt get logged when the capability is assigned, it is logged when the capability is attempted to be used. 

info=denied means they werent successful running |delete

info=granted means they were successful.

Hopefully this clears it up.

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

0 Karma

BB2
Explorer

I understand this better now.

Using the search you provided will search for anyone who did delete a record.  I can then go into Splunk and search the timeframe for who actually performed the delete.

index=_audit action=delete_by_keyword info=granted

I can setup an alert using this search for any failure attempts to delete a record.

index=_audit action=search info=failed | regex search="\\|(\\s|\\n|\\r|([\\s\\S]*))*delete"

0 Karma

BB2
Explorer

I am curious then why I do not get any results if I search for info=denied.  I had attempted to delete a record several times and got the insufficient privileges but nothing showed up in the audit.log.

index=_audit action=delete_by_keyword info=denied

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...