Hello,
A few days ago I had a problem with an index.
The index_size_max
was equal to the index_size
, with the default setting in the indexes.conf file.
Here is the request I used:
| rest /services/data/indexes | where disabled = 0 | search NOT title = "_*" | eval currentDBSizeGB = round( currentDBSizeMB / 1024) | where currentDBSizeGB > 0 | table splunk_server title summaryHomePath_expanded minTime maxTime currentDBSizeGB totalEventCount frozenTimePeriodInSecs coldToFrozenDir maxTotalDataSizeMB | rename minTime AS earliest maxTime AS latest summaryHomePath_expanded AS index_path currentDBSizeGB AS index_size totalEventCount AS event_cnt frozenTimePeriodInSecs AS index_retention coldToFrozenDir AS index_path_frozen maxTotalDataSizeMB AS index_size_max title AS index
On May 14th AM =>
-index_max_size set to 512Go
-index_size = 500Go
-latest data age was "uptodate"
-earliest data age was March 19th - 05:11:30
On May 14th PM =>
-index_max_size set to 1536Go (updated)
-index_size = 509Go
-latest data age was "uptodate"
-earliest data age was March 19th - 05:11:30 (still the same date)
On May 18th AM =>
-index_max_size set to 1536Go
-index_size = 524Go
-latest data age was "uptodate"
-earliest data age was March 19th - 05:11:30 (still the same date)
On May 23th AM =>
-index_max_size set to 1536Go
-index_size = 563Go
-latest data age was "uptodate"
-earliest data age was March 23th - 12:22:28 (not anymore the same date)
On May 26th AM => (today)
-index_max_size set to 1536Go
-index_size = 564Go
-latest data age was "uptodate"
-earliest data age was March 28th - 06:46:27 (not anymore the same date)
Since I've increased the maxTotalDataSizeMB
in indexes.conf, I'm still losing the oldest data, but the index is bigger days after days.
I also notice that the earliest data ages are not exactly the same between my 2 indexers in my cluster.
By default I must keep 1 year of data, and parameters are set for, aka " frozenTimePeriodInSecs = 31557600
"
Can anyone help me please?
Thanks a lot.
P.S. Can someone explain to me why this request gives me information for 2 of 3 indexes I've got?
index names are csmsi_supervision_
followed by active
, passive
or servicenow
.
"passive" is missing.
Thanks.
here are the stanzas:
file: $SPLUNK_HOME/etc/slave-apps/csmsi_all_indexes/local/indexes.conf
[csmsi_supervision_active]
coldPath = volume:cold/csmsi_supervision_active/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = volume:live/csmsi_supervision_active/db
maxTotalDataSizeMB = 1536000
thawedPath = /r3c/r3cadmaa/thawed/csmsi_supervision_active/thaweddb
repFactor = auto
frozenTimePeriodInSecs = 31557600
[csmsi_supervision_servicenow]
coldPath = volume:cold/csmsi_supervision_servicenow/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = volume:live/csmsi_supervision_servicenow/db
maxTotalDataSizeMB = 512000
thawedPath = /r3c/r3cadmaa/thawed/csmsi_supervision_servicenow/thaweddb
repFactor = auto
frozenTimePeriodInSecs = 31557600
[csmsi_supervision_passive]
coldPath = volume:cold/csmsi_supervision_passive/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = volume:live/csmsi_supervision_passive/db
maxTotalDataSizeMB = 512000
thawedPath = /r3c/r3cadmaa/thawed/csmsi_supervision_passive/thaweddb
frozenTimePeriodInSecs = 31557600
repFactor = auto
file : $SPLUNK_HOME/etc/system/default/indexes.conf
maxDataSize = auto
maxWarmDBCount = 300
frozenTimePeriodInSecs = 188697600
rotatePeriodInSecs = 60
coldToFrozenScript =
coldToFrozenDir =
compressRawdata = true
maxTotalDataSizeMB = 500000
maxMemMB = 5
maxConcurrentOptimizes = 6
maxHotSpanSecs = 7776000
maxHotIdleSecs = 0
maxHotBuckets = 3
minHotIdleSecsBeforeForceRoll = auto
quarantinePastSecs = 77760000
quarantineFutureSecs = 2592000
rawChunkSizeBytes = 131072
minRawFileSyncSecs = disable
assureUTF8 = false
serviceMetaPeriod = 25
partialServiceMetaPeriod = 0
throttleCheckPeriod = 15
syncMeta = true
maxMetaEntries = 1000000
maxBloomBackfillBucketAge = 30d
enableOnlineBucketRepair = true
enableDataIntegrityControl = false
maxTimeUnreplicatedWithAcks = 60
maxTimeUnreplicatedNoAcks = 300
minStreamGroupQueueSize = 2000
warmToColdScript=
tstatsHomePath = volume:_splunk_summaries/$_index_name/datamodel_summary
homePath.maxDataSizeMB = 0
coldPath.maxDataSizeMB = 0
streamingTargetTsidxSyncPeriodMsec = 5000
journalCompression = gzip
enableTsidxReduction = false
suspendHotRollByDeleteQuery = false
tsidxReductionCheckPeriodInSec = 600
timePeriodInSecBeforeTsidxReduction = 604800
Thanks for helping
Please share the indexes.conf stanza for the index in question. Please also share the [default]
stanza.