Monitoring Splunk

Can I limit the disk size of a Splunk instance to 300 GB within config files?

natalienguyen
Explorer

I'm looking to set up a stand-alone test Splunk instance and want to limit the disk size of the instance to 300GB.

Is this possible to do within the config files? Or do I need to install it on a separate partition that has 300GB and just let it run?

0 Karma

inventsekar
SplunkTrust
SplunkTrust

http://docs.splunk.com/Documentation/Splunk/7.0.0/Indexer/Setaretirementandarchivingpolicy

Freeze data when an index grows too large: Set maxTotalDataSizeMB
You can use the size of an index to determine when data gets frozen and removed from the index. If an index grows larger than its maximum specified size, the oldest data is rolled to the frozen state.

The default maximum size for an index is 500,000MB. To change the maximum size, edit the maxTotalDataSizeMB attribute in indexes.conf. For example, to specify the maximum size as 250,000MB:

[main]
maxTotalDataSizeMB = 250000

Specify the size in megabytes.

Restart the indexer for the new setting to take effect. Depending on how much data there is to process, it can take some time for the indexer to begin to move buckets out of the index to conform to the new policy. You might see high CPU usage during this time.

0 Karma

anthonymelita
Contributor

Actually not setting the index size smaller than total disk space might inadvertently do what you want. If you set the max size on the index it will roll out the oldest events when the limit is reached. If you run out of disk space it will cause a system alarm and stop indexing. Example: "skipped indexing of internal audit events will keep dropping events until indexer congestion is remedied. Check space and other issues that may caused indexer to block"
Of course this is a symptom, not a solution to your request.

0 Karma

natalienguyen
Explorer

Thanks but this is for an index, I would like the whole instance not to exceed 300GB.

For instance, I could have 10 indexes, but once the total space of them reaches 300GB, then Splunk will stop indexing.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...