Can I delete the DB data files in the search head or are there some steps that I need to follow beforehand?

[_pravin]

Hi Community,

We have a cluster setup for our Splunk install where all the data are indexed at the data layer (data from heavy forwarders, indexers, and even the _internal data from the search head). The current size of indexes in the Splunk Search head should be 1 MB but I notice that one of the indexes and a few internal indexes get data. This increases the size of the search head in addition to the increase in the size of the indexes in the indexers.

When I check the last event received by the indexer in the search head, it shows 8 months ago in the GUI and also in the backend files. But when I check the same in any of the indexers, the last event was received recently. My doubt is can I delete the DB data files in the search head or are there some steps that I need to follow before I remove the DB files directly?

Regards,

Pravin

[gcusello]

Hi @_pravin,

if you have an Indexer Cluster, the easiest way to clean an index is to modify the retention period of that index, modifying the frozenTimePeriodInSecs to a very little value in the Master Node and then push the configuration to the Search Peers (Indexers); in this way all the buckets, except the Hot ones, will be deleted.

Then you can restore the correct retention period.

The thing I don't understand is why you have indexed data on the Search Heads: it's a best practice to forward all logs from the Splunk infrastructure ato Indexers.

Ciao.

Giuseppe

[_pravin]

Hi @gcusello ,

Thanks for the response. I could use this solution to remove data older than 8 months and then revert back to the old configurations.

Even I have the same doubt as to why the search head has data when all the logs have been forwarded to the indexers.

Regards,

Pravin

[gcusello]

Hi @_pravin,

check if there's an index and forward option in outputs.conf.

This is the only option that justifies the presence of new logs on SH.

then check if the indexes on SH are old: maybe they where created in the beginning of the activity and they aren't used when you forwarded logs to Indexers.

Ciao.

Giuseppe

[sylim_splunk · Accepted solution]

You can use "splunk clean eventdata -index " command like below;

$ splunk stop

$ splunk clean eventdata -index "_internal"

[_pravin]

Hi @sylim_splunk ,

Won't this delete the entire data from the index in the cluster? I want to delete only the data present in the search head.

Regards,

Pravin

[sylim_splunk]

@_pravin  the command is only effective on the SH you run. And the index buckets in the indexers are not affected by the commend.

[richgalloway]

If you no longer care about the 8-months-old data on the search then you can delete it.  Be sure to restart the SH afterwards.

[_pravin]

Hi @richgalloway ,

Do you mean to say that I can directly delete the data from the DB and still retain all the data from the indexers? Won't this affect the data in the indexers in any way?

Regards,

Pravin

[richgalloway]

If you delete the DB files from the Search Head, the indexers will not be affected.