topic Re: Freeing up Splunk system disk space in Monitoring Splunk

Is it possible to free up Splunk system disk space?

DCUsupport — Wed, 15 Mar 2023 16:24:13 GMT

Hello everyone,

I know it's possible to remove things from Splunk search that are older than two years, for example. If I apply this setting, space is not freed on the system disk where Splunk is installed.
Therefore, I am asking for information on how to delete data older than two years from Splunk DB, so as to free up space on the system disk.
Is it even possible?
Thank you.

Best Regards,

DCUsupport

Re: Freeing up Splunk system disk space

gcusello — Tue, 14 Mar 2023 13:14:20 GMT

Hi @DCUsupport,

by default, retention is defined in 6 years, but you can configure retention adding the "frozenTimePeriodInSecs" option to the indexes to reduce (in indexes.conf).

I hint to start from _internal that's very verbose and old data aren't so useful.

But anyway, you should configure retention for all your indexes.

Ciao.

Giuseppe

Re: Freeing up Splunk system disk space

PickleRick — Tue, 14 Mar 2023 15:23:12 GMT

1. Verify what is using up your space

2. If these are indexes, check their parameters and possibly lower the limits (time limit for freezing and size limits for indexes; you can also define volume and set overall limit for the whole volume)

3. If these are not indexes, check where they come from. Course of action will depend on what it is.

Re: Freeing up Splunk system disk space

DCUsupport — Fri, 17 Mar 2023 09:29:29 GMT

Hello PickleRick,

occupied space size in KB:

296400092 KB /opt/splunk/var

292801716 KB /opt/splunk/var/lib/splunk

4340556 /opt/splunk/var/lib/splunk/audit

64037852 /opt/splunk/var/lib/splunk/defaultdb

468520 /opt/splunk/var/lib/splunk/fishbucket

198205448 /opt/splunk/var/lib/splunk/fortigate

3017864 /opt/splunk/var/lib/splunk/_internaldb

2622968 /opt/splunk/var/lib/splunk/_introspection

596772 /opt/splunk/var/lib/splunk/kvstore

851316 /opt/splunk/var/lib/splunk/_metrics

18631936 /opt/splunk/var/lib/splunk/os

I don't know how to find out if they are indexes or not.
I am asking for information on how to recognize this, or where I should reduce the limits for indexes. I could not find indexes.conf.

Thank you.

Re: Is it possible to free up Splunk system disk space?

DCUsupport — Fri, 17 Mar 2023 09:40:02 GMT

Hello @PickleRick and @gcusello,

I found the indexes.conf, but I'm not very smart about setting it up. It probably looks like the retention period is 2 years if I understand correctly. But I don't know how to proceed further.
The full listing of indexes.conf can be found below.

#Version 8.2.3
#DO NOT EDIT THIS FILE!
# Changes to default files will be lost on update and are difficult to
# manage and support.
#
# Please make any changes to system defaults by overriding them in
# apps or $SPLUNK_HOME/etc/system/local
# (See "Configuration file precedence" in the web documentation).
#
# To override a specific setting, copy the name of the stanza and
# setting to the file where you wish to override it.
#
# This file configures Splunk's indexes and their properties.
#

################################################################################
# "global" params (not specific to individual indexes)
################################################################################
sync = 0
indexThreads = auto
memPoolMB = auto
defaultDatabase = main
enableRealtimeSearch = true
suppressBannerList =
maxRunningProcessGroups = 8
maxRunningProcessGroupsLowPriority = 1
bucketRebuildMemoryHint = auto
serviceOnlyAsNeeded = true
serviceSubtaskTimingPeriod = 30
serviceInactiveIndexesPeriod = 60
maxBucketSizeCacheEntries = 0
processTrackerServiceInterval = 1
hotBucketTimeRefreshInterval = 10
rtRouterThreads = 0
rtRouterQueueSize = 10000
selfStorageThreads = 2
fileSystemExecutorWorkers = 5
hotBucketStreaming.extraBucketBuildingCmdlineArgs =

################################################################################
# index specific defaults
################################################################################
maxDataSize = auto
maxWarmDBCount = 300
frozenTimePeriodInSecs = 188697600
rotatePeriodInSecs = 60
coldToFrozenScript =
coldToFrozenDir =
compressRawdata = true
maxTotalDataSizeMB = 500000
maxGlobalRawDataSizeMB = 0
maxGlobalDataSizeMB = 0
maxMemMB = 5
maxConcurrentOptimizes = 6
maxHotSpanSecs = 7776000
maxHotIdleSecs = 0
maxHotBuckets = auto
metric.maxHotBuckets = auto
minHotIdleSecsBeforeForceRoll = auto
quarantinePastSecs = 77760000
quarantineFutureSecs = 2592000
rawChunkSizeBytes = 131072
minRawFileSyncSecs = disable
assureUTF8 = false
serviceMetaPeriod = 25
partialServiceMetaPeriod = 0
throttleCheckPeriod = 15
syncMeta = true
maxMetaEntries = 1000000
maxBloomBackfillBucketAge = 30d
enableOnlineBucketRepair = true
enableDataIntegrityControl = false
maxTimeUnreplicatedWithAcks = 60
maxTimeUnreplicatedNoAcks = 300
minStreamGroupQueueSize = 2000
warmToColdScript=
tstatsHomePath = volume:_splunk_summaries/$_index_name/datamodel_summary
homePath.maxDataSizeMB = 0
coldPath.maxDataSizeMB = 0
streamingTargetTsidxSyncPeriodMsec = 5000
journalCompression = gzip
enableTsidxReduction = false
suspendHotRollByDeleteQuery = false
tsidxReductionCheckPeriodInSec = 600
timePeriodInSecBeforeTsidxReduction = 604800
datatype = event
splitByIndexKeys =
metric.splitByIndexKeys =
tsidxWritingLevel = 2
archiver.enableDataArchive = false
archiver.maxDataArchiveRetentionPeriod = 0
hotBucketStreaming.sendSlices = false
hotBucketStreaming.removeRemoteSlicesOnRoll = false
hotBucketStreaming.reportStatus = false
hotBucketStreaming.deleteHotsAfterRestart = false
tsidxDedupPostingsListMaxTermsLimit = 8388608
tsidxTargetSizeMB = 1500
metric.tsidxTargetSizeMB = 1500
metric.enableFloatingPointCompression = true
metric.compressionBlockSize = 1024
metric.stubOutRawdataJournal = true
metric.timestampResolution = s
waitPeriodInSecsForManifestWrite = 60
bucketMerging = false
bucketMerge.minMergeSizeMB = 750
bucketMerge.maxMergeSizeMB = 1000
bucketMerge.maxMergeTimeSpanSecs = 7776000

#
# By default none of the indexes are replicated.
#
repFactor = 0

# Splunk to Splunk federated index
federated.provider =
federated.dataset =

[volume:_splunk_summaries]
path = $SPLUNK_DB

[provider-family:hadoop]
vix.mode = report
vix.command = $SPLUNK_HOME/bin/jars/sudobash
vix.command.arg.1 = $HADOOP_HOME/bin/hadoop
vix.command.arg.2 = jar
vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-h1.jar
vix.command.arg.4 = com.splunk.mr.SplunkMR
vix.env.MAPREDUCE_USER =
vix.env.HADOOP_HEAPSIZE = 512
vix.env.HADOOP_CLIENT_OPTS = -XX:ParallelGCThreads=4 -XX:+UseParallelGC -XX:+DisplayVMOutputToStderr
vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.21.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.10.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-exec-0.12.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-metastore-0.12.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-serde-0.12.0.jar
vix.mapred.job.reuse.jvm.num.tasks = 100
vix.mapred.child.java.opts = -server -Xmx512m -XX:ParallelGCThreads=4 -XX:+UseParallelGC -XX:+DisplayVMOutputToStderr
vix.mapred.reduce.tasks = 0
vix.mapred.job.map.memory.mb = 2048
vix.mapred.job.reduce.memory.mb = 512
vix.mapred.job.queue.name = default
vix.mapreduce.job.jvm.numtasks = 100
vix.mapreduce.map.java.opts = -server -Xmx512m -XX:ParallelGCThreads=4 -XX:+UseParallelGC -XX:+DisplayVMOutputToStderr
vix.mapreduce.reduce.java.opts = -server -Xmx512m -XX:ParallelGCThreads=4 -XX:+UseParallelGC -XX:+DisplayVMOutputToStderr
vix.mapreduce.job.reduces = 0
vix.mapreduce.map.memory.mb = 2048
vix.mapreduce.reduce.memory.mb = 512
vix.mapreduce.job.queuename = default
vix.splunk.search.column.filter = 1
vix.splunk.search.mixedmode = 1
vix.splunk.search.debug = 0
vix.splunk.search.mr.maxsplits = 10000
vix.splunk.search.mr.minsplits = 100
vix.splunk.search.mr.splits.multiplier = 10
vix.splunk.search.mr.poll = 2000
vix.splunk.search.recordreader = SplunkJournalRecordReader,ValueAvroRecordReader,SimpleCSVRecordReader,SequenceFileRecordReader
vix.splunk.search.recordreader.avro.regex = \.avro$
vix.splunk.search.recordreader.csv.regex = \.([tc]sv)(?:\.(?:gz|bz2|snappy))?$
vix.splunk.search.recordreader.sequence.regex = \.seq$
vix.splunk.home.datanode = /tmp/splunk/$SPLUNK_SERVER_NAME/
vix.splunk.heartbeat = 1
vix.splunk.heartbeat.threshold = 60
vix.splunk.heartbeat.interval = 1000
vix.splunk.setup.onsearch = 1
vix.splunk.setup.package = current

################################################################################
# index definitions
################################################################################

[main]
homePath = $SPLUNK_DB/defaultdb/db
coldPath = $SPLUNK_DB/defaultdb/colddb
thawedPath = $SPLUNK_DB/defaultdb/thaweddb
tstatsHomePath = volume:_splunk_summaries/defaultdb/datamodel_summary
maxMemMB = 20
maxConcurrentOptimizes = 6
maxHotIdleSecs = 86400
maxHotBuckets = 10
maxDataSize = auto_high_volume

[history]
homePath = $SPLUNK_DB/historydb/db
coldPath = $SPLUNK_DB/historydb/colddb
thawedPath = $SPLUNK_DB/historydb/thaweddb
tstatsHomePath = volume:_splunk_summaries/historydb/datamodel_summary
maxDataSize = 10
frozenTimePeriodInSecs = 604800

[summary]
homePath = $SPLUNK_DB/summarydb/db
coldPath = $SPLUNK_DB/summarydb/colddb
thawedPath = $SPLUNK_DB/summarydb/thaweddb
tstatsHomePath = volume:_splunk_summaries/summarydb/datamodel_summary

[_internal]
homePath = $SPLUNK_DB/_internaldb/db
coldPath = $SPLUNK_DB/_internaldb/colddb
thawedPath = $SPLUNK_DB/_internaldb/thaweddb
tstatsHomePath = volume:_splunk_summaries/_internaldb/datamodel_summary
maxDataSize = 1000
maxHotSpanSecs = 432000
frozenTimePeriodInSecs = 2592000

[_audit]
homePath = $SPLUNK_DB/audit/db
coldPath = $SPLUNK_DB/audit/colddb
thawedPath = $SPLUNK_DB/audit/thaweddb
tstatsHomePath = volume:_splunk_summaries/audit/datamodel_summary

[_thefishbucket]
homePath = $SPLUNK_DB/fishbucket/db
coldPath = $SPLUNK_DB/fishbucket/colddb
thawedPath = $SPLUNK_DB/fishbucket/thaweddb
tstatsHomePath = volume:_splunk_summaries/fishbucket/datamodel_summary
maxDataSize = 500
frozenTimePeriodInSecs = 2419200

# this index has been removed in the 4.1 series, but this stanza must be
# preserved to avoid displaying errors for users that have tweaked the index's
# size/etc parameters in local/indexes.conf.
#
[splunklogger]
homePath = $SPLUNK_DB/splunklogger/db
coldPath = $SPLUNK_DB/splunklogger/colddb
thawedPath = $SPLUNK_DB/splunklogger/thaweddb
disabled = true

[_introspection]
homePath = $SPLUNK_DB/_introspection/db
coldPath = $SPLUNK_DB/_introspection/colddb
thawedPath = $SPLUNK_DB/_introspection/thaweddb
maxDataSize = 1024
frozenTimePeriodInSecs = 1209600

[_telemetry]
homePath = $SPLUNK_DB/_telemetry/db
coldPath = $SPLUNK_DB/_telemetry/colddb
thawedPath = $SPLUNK_DB/_telemetry/thaweddb
maxDataSize = 256
frozenTimePeriodInSecs = 63072000

[_metrics]
homePath = $SPLUNK_DB/_metrics/db
coldPath = $SPLUNK_DB/_metrics/colddb
thawedPath = $SPLUNK_DB/_metrics/thaweddb
datatype = metric
#14 day retention
frozenTimePeriodInSecs = 1209600
metric.splitByIndexKeys = metric_name

# Internal Use Only: rollup data from the _metrics index.
[_metrics_rollup]
homePath = $SPLUNK_DB/_metrics_rollup/db
coldPath = $SPLUNK_DB/_metrics_rollup/colddb
thawedPath = $SPLUNK_DB/_metrics_rollup/thaweddb
datatype = metric
# 2 year retention
frozenTimePeriodInSecs = 63072000
metric.splitByIndexKeys = metric_name

Re: Is it possible to free up Splunk system disk space?

gcusello — Fri, 17 Mar 2023 11:37:06 GMT

Hi @DCUsupport,

from the above indexer.conf, list the indexes to reduce.

then create a file in a local folder of your app containing the index names and for each one the option 'frozenTimePeriodInSecs' .

in other words to have a retention of 30 days for each index you have to setup:

[index1] frozenTimePeriodInSecs = 2592000 [index2] frozenTimePeriodInSecs = 2592000 [index3] frozenTimePeriodInSecs = 2592000

Ciao.

Giuseppe

Re: Freeing up Splunk system disk space

PickleRick — Fri, 17 Mar 2023 13:39:54 GMT

Typically this directory indeed stores indexes. In your case most of the space is used by the fortigate directory (which probably contains an index called, surprise surprise, fortigate) and the defaultdb directory which contains the main index.

1. Don't touch system/default/indexes.conf! Actually, never touch any system/default/ files. If you need to do config adjustments, put them in your app or system/local.

2. Using the main index isn't a very good practice. You should have purpose-created indexes for your data.

3. As @gcusello already pointed out, the default retention period is 6 years so you might want to tweak that. I suppose you use an all-in-one installation so you can change it in Settings->Indexes. You can also limit the index by size. But remember that the index directory contains not only index data but can contain - for example - accelerated summaries so that size limiting will not be 100% precise (it isn't anyway XD)

Re: Freeing up Splunk system disk space

DCUsupport — Fri, 17 Mar 2023 16:04:46 GMT

Hello @PickleRick and @gcusello

In settings -> indexes I found this, but I don't see the option to adjust retention.

I tried looking for config for Fortigate and only found app.conf, TA-FortinetAR.conf, TA-FortinetAR_credential.conf, TA-FortinetAR_customized.conf in /opt/splunk/etc/apps/TA-FortinetAR/local. All contain only the default parameter, except for app.conf which contains:
[install]
state = disabled
is_configured = 0

Is it enough to add the parameter frozenTimePeriodInSecs with its own value to TA-FortinetAR.conf?

Or it's enough for me in indexes.conf
add this:

[fortigate]
homePath = $SPLUNK_DB/fortigate/db
coldPath = $SPLUNK_DB/fortigate/colddb
thawedPath = $SPLUNK_DB/fortigate/thaweddb
maxDataSize = auto_high_volume
frozenTimePeriodInSecs = 63072000

or something similar?
If it will be enough and I will set this retention. Will disk space be automatically freed up if there is currently more than two and a half years of data there and the retention will be for two years?

Again, sorry if my questions are wrong. I don't have much experience with Splunk yet.

Thank you.

Re: Freeing up Splunk system disk space

gcusello — Fri, 17 Mar 2023 16:07:30 GMT

Hi @DCUsupport,

I don't know why this parameter is configurable only by conf file and not by GUI, so you have to add the row in each stanza of your indexes.conf file.

As @PickleRick said, don't modify conf files in default folders: copy and modify them in local folders.

Ciao.

Giuseppe

Re: Freeing up Splunk system disk space

PickleRick — Fri, 17 Mar 2023 16:11:12 GMT

Hah. I haven't configured indexes via gui for so long that I forgot that indeed the time to frozen is not set there.

The app you mentioned is most probably responsible for parsing the data and maybe displaying it in some dashboards so don't touch it.

Depending on where your fortigate index is defined, it's probably most convenient to adjust the frozenTimePeriodInSecs in the same file.

Just do

splunk btool indexes list fortigate --debug

It will show you in which file(s) settings regarding this file are defined. If there is already the frozenTimePeriodInSecs setting, just edit the file where it's specified and adjust it. If it is not, just edit the file where the setting is defined (don't touch anything from system/default directory!). And add this setting.

Re: Freeing up Splunk system disk space

DCUsupport — Fri, 17 Mar 2023 17:03:50 GMT

Hello @PickleRick and @gcusello

the splunk btool indexes list fortigate --debug command listing below:

/opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf [fortigate]
/opt/splunk/etc/system/default/indexes.conf archiver.enableDataArchive = false
/opt/splunk/etc/system/default/indexes.conf archiver.maxDataArchiveRetentionPeriod = 0
/opt/splunk/etc/system/default/indexes.conf assureUTF8 = false
/opt/splunk/etc/system/default/indexes.conf bucketMerge.maxMergeSizeMB = 1000
/opt/splunk/etc/system/default/indexes.conf bucketMerge.maxMergeTimeSpanSecs = 7776000
/opt/splunk/etc/system/default/indexes.conf bucketMerge.minMergeSizeMB = 750
/opt/splunk/etc/system/default/indexes.conf bucketMerging = false
/opt/splunk/etc/system/default/indexes.conf bucketRebuildMemoryHint = auto
/opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf coldPath = $SPLUNK_DB/fortigate/colddb
/opt/splunk/etc/system/default/indexes.conf coldPath.maxDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf coldToFrozenDir =
/opt/splunk/etc/system/default/indexes.conf coldToFrozenScript =
/opt/splunk/etc/system/default/indexes.conf compressRawdata = true
/opt/splunk/etc/system/default/indexes.conf datatype = event
/opt/splunk/etc/system/default/indexes.conf defaultDatabase = main
/opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf enableDataIntegrityControl = 0
/opt/splunk/etc/system/default/indexes.conf enableOnlineBucketRepair = true
/opt/splunk/etc/system/default/indexes.conf enableRealtimeSearch = true
/opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf enableTsidxReduction = 0
/opt/splunk/etc/system/default/indexes.conf federated.dataset =
/opt/splunk/etc/system/default/indexes.conf federated.provider =
/opt/splunk/etc/system/default/indexes.conf fileSystemExecutorWorkers = 5
/opt/splunk/etc/system/default/indexes.conf frozenTimePeriodInSecs = 188697600
/opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf homePath = $SPLUNK_DB/fortigate/db
/opt/splunk/etc/system/default/indexes.conf homePath.maxDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf hotBucketStreaming.deleteHotsAfterRestart = false
/opt/splunk/etc/system/default/indexes.conf hotBucketStreaming.extraBucketBuildingCmdlineArgs =
/opt/splunk/etc/system/default/indexes.conf hotBucketStreaming.removeRemoteSlicesOnRoll = false
/opt/splunk/etc/system/default/indexes.conf hotBucketStreaming.reportStatus = false
/opt/splunk/etc/system/default/indexes.conf hotBucketStreaming.sendSlices = false
/opt/splunk/etc/system/default/indexes.conf hotBucketTimeRefreshInterval = 10
/opt/splunk/etc/system/default/indexes.conf indexThreads = auto
/opt/splunk/etc/system/default/indexes.conf journalCompression = gzip
/opt/splunk/etc/system/default/indexes.conf maxBloomBackfillBucketAge = 30d
/opt/splunk/etc/system/default/indexes.conf maxBucketSizeCacheEntries = 0
/opt/splunk/etc/system/default/indexes.conf maxConcurrentOptimizes = 6
/opt/splunk/etc/system/default/indexes.conf maxDataSize = auto
/opt/splunk/etc/system/default/indexes.conf maxGlobalDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf maxGlobalRawDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf maxHotBuckets = auto
/opt/splunk/etc/system/default/indexes.conf maxHotIdleSecs = 0
/opt/splunk/etc/system/default/indexes.conf maxHotSpanSecs = 7776000
/opt/splunk/etc/system/default/indexes.conf maxMemMB = 5
/opt/splunk/etc/system/default/indexes.conf maxMetaEntries = 1000000
/opt/splunk/etc/system/default/indexes.conf maxRunningProcessGroups = 8
/opt/splunk/etc/system/default/indexes.conf maxRunningProcessGroupsLowPriority = 1
/opt/splunk/etc/system/default/indexes.conf maxTimeUnreplicatedNoAcks = 300
/opt/splunk/etc/system/default/indexes.conf maxTimeUnreplicatedWithAcks = 60
/opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf maxTotalDataSizeMB = 512000
/opt/splunk/etc/system/default/indexes.conf maxWarmDBCount = 300
/opt/splunk/etc/system/default/indexes.conf memPoolMB = auto
/opt/splunk/etc/system/default/indexes.conf metric.compressionBlockSize = 1024
/opt/splunk/etc/system/default/indexes.conf metric.enableFloatingPointCompression = true
/opt/splunk/etc/system/default/indexes.conf metric.maxHotBuckets = auto
/opt/splunk/etc/system/default/indexes.conf metric.splitByIndexKeys =
/opt/splunk/etc/system/default/indexes.conf metric.stubOutRawdataJournal = true
/opt/splunk/etc/system/default/indexes.conf metric.timestampResolution = s
/opt/splunk/etc/system/default/indexes.conf metric.tsidxTargetSizeMB = 1500
/opt/splunk/etc/system/default/indexes.conf minHotIdleSecsBeforeForceRoll = auto
/opt/splunk/etc/system/default/indexes.conf minRawFileSyncSecs = disable
/opt/splunk/etc/system/default/indexes.conf minStreamGroupQueueSize = 2000
/opt/splunk/etc/system/default/indexes.conf partialServiceMetaPeriod = 0
/opt/splunk/etc/system/default/indexes.conf processTrackerServiceInterval = 1
/opt/splunk/etc/system/default/indexes.conf quarantineFutureSecs = 2592000
/opt/splunk/etc/system/default/indexes.conf quarantinePastSecs = 77760000
/opt/splunk/etc/system/default/indexes.conf rawChunkSizeBytes = 131072
/opt/splunk/etc/system/default/indexes.conf repFactor = 0
/opt/splunk/etc/system/default/indexes.conf rotatePeriodInSecs = 60
/opt/splunk/etc/system/default/indexes.conf rtRouterQueueSize = 10000
/opt/splunk/etc/system/default/indexes.conf rtRouterThreads = 0
/opt/splunk/etc/system/default/indexes.conf selfStorageThreads = 2
/opt/splunk/etc/system/default/indexes.conf serviceInactiveIndexesPeriod = 60
/opt/splunk/etc/system/default/indexes.conf serviceMetaPeriod = 25
/opt/splunk/etc/system/default/indexes.conf serviceOnlyAsNeeded = true
/opt/splunk/etc/system/default/indexes.conf serviceSubtaskTimingPeriod = 30
/opt/splunk/etc/system/default/indexes.conf splitByIndexKeys =
/opt/splunk/etc/system/default/indexes.conf streamingTargetTsidxSyncPeriodMsec = 5000
/opt/splunk/etc/system/default/indexes.conf suppressBannerList =
/opt/splunk/etc/system/default/indexes.conf suspendHotRollByDeleteQuery = false
/opt/splunk/etc/system/default/indexes.conf sync = 0
/opt/splunk/etc/system/default/indexes.conf syncMeta = true
/opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf thawedPath = $SPLUNK_DB/fortigate/thaweddb
/opt/splunk/etc/system/default/indexes.conf throttleCheckPeriod = 15
/opt/splunk/etc/system/default/indexes.conf timePeriodInSecBeforeTsidxReduction = 604800
/opt/splunk/etc/system/default/indexes.conf tsidxDedupPostingsListMaxTermsLimit = 8388608
/opt/splunk/etc/system/default/indexes.conf tsidxReductionCheckPeriodInSec = 600
/opt/splunk/etc/system/default/indexes.conf tsidxTargetSizeMB = 1500
/opt/splunk/etc/system/default/indexes.conf tsidxWritingLevel = 2
/opt/splunk/etc/system/default/indexes.conf tstatsHomePath = volume:_splunk_summaries/$_index_name/datamodel_summary
/opt/splunk/etc/system/default/indexes.conf waitPeriodInSecsForManifestWrite = 60
/opt/splunk/etc/system/default/indexes.conf warmToColdScript =

cat /opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf
[fortigate]
coldPath = $SPLUNK_DB/fortigate/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/fortigate/db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/fortigate/thaweddb

Should it be enough to add the parameter frozenTimePeriodInSecs to /opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf ?
Because it is not populated in /opt/splunk/etc/apps/SplunkAppForFortinet/local/indexes.conf and it seems to take this value from system/default where you have forbidden me to make modifications which I understand 🙂

Thank you for your patience in resolving this issue.

Re: Freeing up Splunk system disk space

PickleRick — Sat, 18 Mar 2023 09:42:59 GMT

Yes, that's where the index seems to be configured (apart from the inherited defaults). I'd put the setting there.

Re: Freeing up Splunk system disk space

DCUsupport — Mon, 20 Mar 2023 15:46:06 GMT

Hello @PickleRick and @gcusello,

It looks like it works.
Thank you so much for your help both.

Re: Freeing up Splunk system disk space

gcusello — Mon, 20 Mar 2023 16:00:17 GMT

Hi @DCUsupport ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉