Getting Data In

Can I delete the DB data files in the search head or are there some steps that I need to follow beforehand?

_pravin
Communicator

Hi Community,

 

We have a cluster setup for our Splunk install where all the data are indexed at the data layer (data from heavy forwarders, indexers, and even the _internal data from the search head). The current size of indexes in the Splunk Search head should be 1 MB but I notice that one of the indexes and a few internal indexes get data. This increases the size of the search head in addition to the increase in the size of the indexes in the indexers.

When I check the last event received by the indexer in the search head, it shows 8 months ago in the GUI and also in the backend files. But when I check the same in any of the indexers, the last event was received recently. My doubt is can I delete the DB data files in the search head or are there some steps that I need to follow before I remove the DB files directly?

 

Regards,

Pravin

Labels (1)
Tags (3)
0 Karma
1 Solution

sylim_splunk
Splunk Employee
Splunk Employee

You can use "splunk clean eventdata -index " command like below;

$ splunk stop

$ splunk clean eventdata -index "_internal"

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @_pravin,

if you have an Indexer Cluster, the easiest way to clean an index is to modify the retention period of that index, modifying the frozenTimePeriodInSecs to a very little value in the Master Node and then push the configuration to the Search Peers (Indexers); in this way all the buckets, except the Hot ones, will be deleted.

Then you can restore the correct retention period.

The thing I don't understand is why you have indexed data on the Search Heads: it's a best practice to forward all logs from the Splunk infrastructure ato Indexers.

Ciao.

Giuseppe

_pravin
Communicator

Hi @gcusello ,

 

Thanks for the response. I could use this solution to remove data older than 8 months and then revert back to the old configurations.

Even I have the same doubt as to why the search head has data when all the logs have been forwarded to the indexers.

 

Regards,

Pravin

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @_pravin,

check if there's an index and forward option in outputs.conf.

This is the only option that justifies the presence of new logs on SH.

then check if the indexes on SH are old: maybe they where created in the beginning of the activity and they aren't used when you forwarded logs to Indexers.

Ciao.

Giuseppe

sylim_splunk
Splunk Employee
Splunk Employee

You can use "splunk clean eventdata -index " command like below;

$ splunk stop

$ splunk clean eventdata -index "_internal"

_pravin
Communicator

Hi @sylim_splunk ,

 

Won't this delete the entire data from the index in the cluster? I want to delete only the data present in the search head.

 

Regards,

Pravin

sylim_splunk
Splunk Employee
Splunk Employee

@_pravin  the command is only effective on the SH you run. And the index buckets in the indexers are not affected by the commend.

richgalloway
SplunkTrust
SplunkTrust

If you no longer care about the 8-months-old data on the search then you can delete it.  Be sure to restart the SH afterwards.

---
If this reply helps you, Karma would be appreciated.

_pravin
Communicator

Hi @richgalloway ,

 

Do you mean to say that I can directly delete the data from the DB and still retain all the data from the indexers? Won't this affect the data in the indexers in any way?

 

Regards,

Pravin

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If you delete the DB files from the Search Head, the indexers will not be affected.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...