I want to delete logs from the last 3 months permanently from each indexer present inside the indexer cluster using a search.
The search below provides me the with the output of the raw logs older than 3 months
source=* sourcetype=* host=* latest=-90d@d earliest=0
Found out that the delete
command doesn't delete the logs completely from the disk and the remove
command cannot be used in an indexer clustering environment.
Do I have to rely only on the bucket rolling parameter set?
Is it necessary to mention each parameter in indexes.conf, or is it enough to mention frozenTimePeriodInSecs =
?
As you note, the |delete command doesnt delete the logs from the buckets. It actually marks them as unsearchable, and then they are deleted based on the retention policy of the index those logs are in.
So as you mention, you can set the frozentimeperiodinseconds to 90 days, and it will roll all your buckets out based on a 90 day retention time. Note this applies to all sources and sourcetypes in an index. Splunk currently doesnt have the ability to age out source/sourcetypes yet in this manner.
This should also be applied from the cluster master server for index cluster, per each index you want to apply this to.