Other Usage

How can I really delete an event

w199284
Explorer

I have read about the "delete" command and used it. However, my security people want certain events gone without the possibility of recovery. I've looked a little at CLI Search with -output table that looks promising. The idea would be to export the index, remove the offending data and re import/index the result. The original source is long gone. Has anybody had to attempt anything similar?

0 Karma

w199284
Explorer

I found a way that worked for me.

• suspend the offending data feed
• run a search that returns the offending data and pipe to |delete
• Take the index offline
• backup the index
• run a shell script (I'm not much of a script jocky) that returns buckets that have "deletes" folders under rawdata
o pass that bucket name to splunk's exporttool and output as a -csv workfile
o pass the csv workfile output to splunk's importtool and create/output a new bucket with the original name "bucketname.new"
o remove the old bucket and rename bucketname.new to bucketname
• put the index back online/test
• resume the data feed

It is a lot of steps, in my case it took 10 hours to complete (mainly waiting on the export/import to finish) and I had to process warm and cold buckets on 12 index peers. I ran these as background tasks. One for warm buckets. One for cold buckets. I performed a lot of tests before turning this loose. In the end, all the data that had been |delete(d) was gone. Since there was about 5 years of history in play, worth the effort.

0 Karma

tpeveler_splunk
Splunk Employee
Splunk Employee

One option would be to use the dump command along with the clean command. After which you would re-index the events.

You would essentially execute a search that identifies the good events and dump them to local disk in raw format. See Splunk dump command in Search Reference.

You would then clean the index of all events via the splunk clean ... CLI command. See Remove data from one or all indexes.

Finally, you would re-index the events that were dumped to disk.

frechetta93
Explorer

@tpeveler_splunk How does one re-index the dumped events?

0 Karma

ChrisG
Splunk Employee
Splunk Employee

The delete search command only marks data for exclusion in subsequent searches.

If you want to remove specific data, you should use the clean CLI command.

If you want to remove an index entirely, use the remove index CLI command.

See Remove indexes and indexed data in the Managing Indexers and Clusters of Indexers manual.

Get Updates on the Splunk Community!

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...