[UPDATE] Hello everyone, and thanks in advance for your help. I'm very new to this subject so if anything is unclear, i'll try to explain my problem more in details. I'm using spunk 9.2.1, and i re...
See more...
[UPDATE] Hello everyone, and thanks in advance for your help. I'm very new to this subject so if anything is unclear, i'll try to explain my problem more in details. I'm using spunk 9.2.1, and i recently observed that my indexer was not indexing logs received. The indexer is in a failure state because my partition $SPLUNK_DB reached the minFreeSpace allowed in server.conf. After further analysis it seems that one of the index _metrics on the partition is saturated with warm buckets (db_*) and taking all the space available. I however have configured all my indexes with the indexes.conf ($SPLUNK_HOME/etc/system/default/indexes.conf) # index specific defaults
maxTotalDataSizeMB = 5000
maxDataSize = 1000
maxMemMB = 5
maxGlobalRawDataSizeMB = 0
maxGlobalDataSizeMB = 0
rotatePeriodInSecs = 30
maxHotIdleSecs = 432000
maxHotSpanSecs = 7776000
maxHotBuckets = auto
maxWarmDBCount = 300
frozenTimePeriodInSecs = 188697600
...
# there's more but i might not be able to disclose them or it might not be revelant
[_metrics]
coldPath = $SPLUNK_DB/_metrics/colddb
homePath = $SPLUNK_DB/_metrics/db
thawedPath = $SPLUNK_DB/_metrics/thaweddb
frozenTimePeriodInSecs = 1209600 From what i understand with this conf applied the index should not exceed 5GB, and when reached the warm/hot buckets should be removed, but it seems that's it's not taken into account in my case. The indexer work fine after purging the buckets and restarting it, but i don't get why the conf was not applied ? Is there something i didn't get here ? Is there a way to check the "characteristics" of my index once started ? -> Checked, the conf is correctly applied. If you know anything on this subject please help me thank you