Will deleting audit events be logged in the internal index?

[just4testsplunk]

I wonder if the activity of deleting audit events from Splunk cloud will be logged/tracked in Splunk internal logs, e.g. logged as sourcetype of splunk_ui_access. If so, is there an official document that clearly states this? Is there any other evidence that someone deleted audit events?

[Aatom]

Unfortunately, we are in the same situation. I don't see an easy solution but here are a few searches that can help. If anyone has a better solution, it would be great to hear.

(This will tell you if someone attempts to delete data w/o the permissions)

index=_internal orig_component="StreamingDeleteOperator" sourcetype=splunk_search_messages | stats count by app message _time | mvcombine message

(This will tell you when someone pipes the delete command into a search)

index=_audit "| delete" search!="'search index=_audit \"| delete\"'"

I have yet to see an audit log for successful deletion.

Always be careful when searching the delete term, as best practice you should only apply the "Can Delete" capability for the period it is needed, and the search results should always be tested before attempting to use.

[KaraD]

Hi @Aatom! Thanks for your community input. Since this is an old post, I recommend starting a new thread with your question, so it can gain more current visibility. 

Cheers!

-Kara D, Splunk Community Manager

[gcusello]

Hi @just4testsplunk_2020,
all the events aren't deleted in Spluk (on premise or cloud) during the retention period until is reached the max dimension of the index: the oldest events are deleted only when they exceed the retention period or the max dimension of the index, if there isn't a coldToFrozenScript that archives them off line.

If you don't see events, check the retention period (usually in Splunk Cloud it's 3 months) and the max dimension of your storage, if you need more storage or more retention you have to buy them.

Ciao.
Giuseppe

[just4testsplunk]

Thanks for your response! Maybe my question is not clear enough. Sorry about that. My question is that if someone with admin role deletes audit events or some other events, e.g. index=audit | delete, this activity will be logged as an event into index=_internal? In other words, if some deletes events by running the delete command, it can be traced and trackable, correct? Thanks.

[ngablern]

Did you ever get an answer on this?  There has to be a way to alert on or audit deletion of logs.

[PickleRick]

OK. Firstly, there should be no need to use the delete command in Splunk. Ever. If the data is properly onboarded, it should not need to be removed. If you need to ingest it into a test index while developing proper ingestion settings, you can index them in a temporary index which rotates quickly.

So no role should need the delete capability under normal circumstances.

Secondly - you can do stuff like monitoring splunk's internal log files (splunk's internal indexes data comes from the logs in $SPLUNK_HOME/var/log) with another solution (like rsyslog or something) but that can be bypassed as well.