topic How to delete data in a filepath on Splunk database? in Getting Data In

How to delete data in a filepath on Splunk database?

dhsetty — Tue, 29 Sep 2020 13:06:06 GMT

Hi all,

I have a Splunk DB search as below:
a=1
b=1000

search_parms = {'date_from': '1/10/2016:05:00', 'start': a, 'stop': b, 'timeout': 60, 'date_to': '02/22/2017:23:39', 'mask_prvs': 0, 'maxresults': 100000},

a. How to delete these queried results from Splunk DB?
b. How to find the Splunk DB storage space after deleted?

Thanks & Regards,
Dharmendra Setty

Re: How to delete data in a filepath on Splunk database?

muebel — Fri, 03 Mar 2017 14:17:35 GMT

Hi dhsetty, You can delete data from a Splunk index by running the delete command after searching for all the data you wish to be deleted.

Note, the delete command won't free up any storage space. It essentially marks those events as unsearchable in the index.

To entirely remove data, you'd have to delete the index, or allow for the retention settings to take care of it (time, disk space, however you have retention set for the index).

Many more details are available here : http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/RemovedatafromSplunk

Please let me know if this answers your question!

Re: How to delete data in a filepath on Splunk database?

richgalloway — Fri, 03 Mar 2017 14:18:03 GMT

Once data has been indexed it cannot be deleted until the bucket ages out. Events can marked, using the delete command, to not appear in search results, but doing so does not change the event and does not save disk space (I believe it uses more space).

Re: How to delete data in a filepath on Splunk database?

dhsetty — Tue, 29 Sep 2020 13:06:15 GMT

Hi Muebel,

I got the below syntax from the Documentation on splunk:

index=fbus_summary latest=1417356000 earliest=1417273200 | eval index = "fbus_summary" | delete

But now the concern is, Iam not getting on how to use the "index" command on my Email Security Appliance. Could you please share your inputs on this?

Thanks & Regards,
Dharmendra Setty

Re: How to delete data in a filepath on Splunk database?

dhsetty — Tue, 29 Sep 2020 13:06:18 GMT

Hi RichGalloway,

I got the below syntax from the Documentation on splunk:

index=fbus_summary latest=1417356000 earliest=1417273200 | eval index = "fbus_summary" | delete

But now the concern is, Iam not getting on how to use the "index" command on my Email Security Appliance. Could you please share your inputs on this?

Thanks & Regards,
Dharmendra Setty

Re: How to delete data in a filepath on Splunk database?

richgalloway — Fri, 03 Mar 2017 14:56:08 GMT

Use the index command in Splunk, not on your ESA. Do not use the index name from the documentation - use the index where the data you want to delete is stored.

Re: How to delete data in a filepath on Splunk database?

muebel — Fri, 03 Mar 2017 16:43:25 GMT

first of all, be very careful with that delete command. Do you have a local splunk certified admin to help? You do not want to throw that command around without careful consideration.

Secondly, that eval statement isn't needed. Based on the search results, all events will have fbus_summary for the index value.

Thirdly, if you do run that command it would delete ALL events in that index for that time frame. You will want to qualify the search to be very specific regarding the events you want deleted.

But now the concern is, Iam not getting on how to use the "index" command on my Email Security Appliance.

I don't quite know what you mean by Email Security Appliance. There isn't an index command in splunk, index is one of the default fields that each event has a value for, and is used in searching.

Re: How to delete data in a filepath on Splunk database?

dhsetty — Mon, 06 Mar 2017 04:13:05 GMT

Hi Richgalloway,

Is there anyway where we can delete the Splunk Data based on timestamp from ESA itself?

Thanks & Regards,
Dharmendra Setty

Re: How to delete data in a filepath on Splunk database?

dhsetty — Mon, 06 Mar 2017 09:26:40 GMT

To add more clarity to my latest query in this thread, about the requirement:

I do a query to splunk, based on time stamp, "from date" & "to date".
After I got the list of all events results between the timestamp, I want to delete these list of events from the Splunk database.
Each queried results data will be stored in the destination database, hence I want to delete each queried results data from querying Splunk DB, so that my next query will not end up in giving repetitive results.

Hence I want a effective solution on how to delete completely the Queried result data, from querying Splunk DB?

Thanks & Regards,
Dharmendra Setty

Re: How to delete data in a filepath on Splunk database?

richgalloway — Mon, 06 Mar 2017 14:50:42 GMT

Once you create a query that returns the events you do not want to see, add | delete to the end of it. That will keep the events from appearing in any subsequent searches. It will NOT delete them from Splunk, however, and there is no way to do so.

Re: How to delete data in a filepath on Splunk database?

dhsetty — Tue, 07 Mar 2017 06:10:53 GMT

Earlier working splunk query is:

SEARCH "mid=" OR "icid=" starttime::12/26/2016:05:05:00 endtime::12/26/2016:05:06:59maxtime::-1 maxevents::100 GET events::0-99 OUTPUT splunkui::2.1

Splunk Query based on your suggestion tried, but it dint work:

SEARCH "mid=" OR "icid=" starttime::12/26/2016:05:05:00 endtime::12/26/2016:05:06:59maxtime::-1 maxevents::100 GET events::0-99 OUTPUT splunkui::2.1 |delete

But it is not fetching any results.

Please let me know what Iam missing here, so that Iam not getting results itself,
I was expecting unique results. But not getting the results only.

Re: How to delete data in a filepath on Splunk database?

dhsetty — Tue, 29 Sep 2020 13:07:14 GMT

Output I got for the Query:

Final Query that is going to _execute_search is:SEARCH "mid=" OR "icid=" starttime::12/26/2016:05:05:00 endtime::12/26/2016:05:06:59maxtime::-1 maxevents::100 GET events::0-99 OUTPUT splunkui::2.1 |delete

API DATA passed to request is:

1
admin
1150854670

1488866453110235SEARCH "mid=" OR "icid=" starttime::12/26/2016:05:05:00 endtime::12/26/2016:05:06:59maxtime::-1 maxevents::100 GET events::0-99 OUTPUT splunkui::2.1 |delete1488866453110275livesplunkuser

results list is :
Queryid=1488866453110235 user=admin result="Search Parse failed because Unrecognized token : |delete" results_returned=0 submitted=03/07/2017:06:00:53 time_between_submission_and_execution=33553144.734 execution_time=1287.266 total_time=0.000
SEARCH "mid=" OR "icid=" starttime::12/26/2016:05:05:00 endtime::12/26/2016:05:06:59maxtime::-1 maxevents::100 GET events::0-99 OUTPUT splunkui::2.1 |delete
Search Parse failed because Unrecognized token : |delete

Re: How to delete data in a filepath on Splunk database?

richgalloway — Tue, 07 Mar 2017 14:26:52 GMT

The delete command does not return events. It only returns a count of the number of events that were deleted.

I'm not familiar enough with the API to help with it.

Re: How to delete data in a filepath on Splunk database?

dhsetty — Tue, 07 Mar 2017 14:45:06 GMT

ok, but still it is considering "delete" as a unrecognized token without even returning the count.