I would like to understand if i have any misconfiguration on my indexes files, and for how long do i keep logs online, archived and when they are deleted (since my HDD is getting full quickly):
[default] suppressBannerList = frozenTimePeriodInSecs = 15778463 throttleCheckPeriod = 15 quarantineFutureSecs = 2592000 partialServiceMetaPeriod = 0 serviceOnlyAsNeeded = true maxHotBuckets = 3 enableOnlineBucketRepair = true bucketRebuildMemoryHint = auto maxRunningProcessGroups = 8 maxDataSize = auto maxWarmDBCount = 300 assureUTF8 = false maxHotIdleSecs = 0 enableRealtimeSearch = true serviceMetaPeriod = 25 repFactor = 0 maxConcurrentOptimizes = 3 maxHotSpanSecs = 7776000 maxTimeUnreplicatedWithAcks = 60 syncMeta = true coldToFrozenDir = maxRunningProcessGroupsLowPriority = 1 serviceSubtaskTimingPeriod = 30 quarantinePastSecs = 77760000 rawChunkSizeBytes = 131072 sync = 0 maxBucketSizeCacheEntries = 1000000 coldToFrozenScript = "/opt/splunk/bin/python" "/opt/splunk/bin/coldToFrozen.py" rotatePeriodInSecs = 60 memPoolMB = auto defaultDatabase = main enableDataIntegrityControl = true minRawFileSyncSecs = disable compressRawdata = true maxMetaEntries = 1000000 maxBloomBackfillBucketAge = 30d maxTotalDataSizeMB = 500000 maxTimeUnreplicatedNoAcks = 300 [_audit] coldPath = $SPLUNK_DB/audit/colddb homePath = $SPLUNK_DB/audit/db thawedPath = $SPLUNK_DB/audit/thaweddb [_internal] frozenTimePeriodInSecs = 2419200 homePath = $SPLUNK_DB/_internaldb/db thawedPath = $SPLUNK_DB/_internaldb/thaweddb maxDataSize = 100 coldPath = $SPLUNK_DB/_internaldb/colddb [_thefishbucket] frozenTimePeriodInSecs = 2419200 homePath = $SPLUNK_DB/fishbucket/db thawedPath = $SPLUNK_DB/fishbucket/thaweddb maxDataSize = 10 coldPath = $SPLUNK_DB/fishbucket/colddb [history] frozenTimePeriodInSecs = 604800 homePath = $SPLUNK_DB/historydb/db thawedPath = $SPLUNK_DB/historydb/thaweddb maxDataSize = 10 coldPath = $SPLUNK_DB/historydb/colddb [main] maxDataSize = auto_high_volume homePath = $SPLUNK_DB/defaultdb/db maxHotBuckets = 10 coldPath = $SPLUNK_DB/defaultdb/colddb maxHotIdleSecs = 86400 maxConcurrentOptimizes = 6 thawedPath = $SPLUNK_DB/defaultdb/thaweddb [splunklogger] coldPath = $SPLUNK_DB/splunklogger/colddb disabled = true homePath = $SPLUNK_DB/splunklogger/db thawedPath = $SPLUNK_DB/splunklogger/thaweddb [summary] coldPath = $SPLUNK_DB/summarydb/colddb homePath = $SPLUNK_DB/summarydb/db thawedPath = $SPLUNK_DB/summarydb/thaweddb
Read these post about how Splunk's data rentention policy works and what all indexes.conf parameters are used in setting them. Once you know about how it's implemented, you'd be able to read and understand your indexes.conf values.
This looks like an exact copy of the default indexes conf with some added/changed values. And you seem to not know what you are doing.
Anyway. So... I'm assuming you are currently storing all your data in the "main" index.
This means that here the [default]
frozenTimePeriodInSecs = 15778463 applies to the retention time. Which is approx. 182 days.
How to fix this:
go to the $SPLUNK_HOME directory (under linux it's /opt/splunk/)
Navigate from there to /opt/splunk/etc/system/local/
Create a file called "indexes.conf"
Write the following:
[main] frozenTimePeriodInSecs = 604800
Save and restart splunk. Now the data in the main index will be saved for only 7 days instead of 182.
If you wanna know more about what indexes.conf does and what the parameters do, look here:
@alvaroveiga, please keep in mind that the twin configuration parameter of
maxTotalDataSizeMB which as we can see on line #39 has the default of 500000 MBs, around 1/2 TB.
Together they control the size of the index.