Reduce the 200G limit in data collection in SII

[rako1980]

For my small number of hosts being monitored by Splunk Insight in Infrastructure, I would like to limit the 200G limit to much lower like 50G or so, so that I think the purging works automatically after it hits the limit. How do I lower that limit? I don't want splunk insight to reach 200G limit.

One thing I thought was just let the SII server to have around 50G disk, but it stops the collection as soon as there is only 5G left with error:
ERROR DiskMon - Disk Monitor: The index processor has paused data flow. Current free disk space on partition '/' has fallen to 4998MB, below the minimum of 5000MB. Data writes to index path '/opt/splunk/var/lib/splunk/em_metrics/db'cannot safely proceed. Increase free disk space on partition '/' by removing or relocating data.

[pwu_splunk]

There's a parameter in indexes.conf that addresses this (maxVolumeDataSizeMB).

https://docs.splunk.com/Documentation/Splunk/7.3.2/Admin/Indexesconf

[rako1980]

@pwu_splunk I have set the parameter as instructed, but I do not see the data shrink.
root@**:~# grep maxVolume /opt/splunk/etc/system/local/indexes.conf
maxVolumeDataSizeMB = 10480
root@*:~# du -hs /opt/splunk/var/lib/splunk
19G /opt/splunk/var/lib/splunk

I have restarted the splunk servoce. So splunkd process was restarted.
I am using the Splunk Insight for Infrastructure and not the Splunk Enterprise.

[pwu_splunk]

Which volume did you add the parameter to?

[rako1980]

Not sure if I understood "which volume". These are the p;aces I added those parameters and restart the spunk:

grep maxVolumeDataSizeMB -R /opt/splunk/etc | grep -v README

/opt/splunk/etc/system/local/indexes.conf:maxVolumeDataSizeMB = 10480
/opt/splunk/etc/apps/splunk_app_infrastructure/default/indexes.conf:maxVolumeDataSizeMB = 10480
/opt/splunk/etc/apps/splunk_app_infrastructure/local/indexes.conf:maxVolumeDataSizeMB = 10480

[twhite_splunk]

hi there @rako1980 - there is more than one index within a given Splunk install - is the goal that you only want Splunk to occupy so much space in general?

[rako1980]

@twhite_splunk My understanding, SII collects data from log and metrics both. SII docs states that the installation includes only 200G of data. I would like to reduce that whole in general to use much lesser than 200G. Note that this is a standalone SII installation

[twhite_splunk]

Hi @rako1980 , could you clarify your request? Are you saying you only want the SII data to occupy a certain amount of space on your indexers, or are you asking how to reduce ingest?

[rako1980]

@twhite_splunk That is correct. Looking for guidance on how can I configure to make SII data to use certain amount of space instead of default 200G. If not, also a way to clean up the SII data to truncate or reduce the s[ace occupied. Thanks you.

[rako1980]

Anyone? I need to have Splunk Insight for Infrastructure only limit to 20G or so instead of 200G default.

[rako1980]

Well, it is probably one of the settings in /opt/splunk/etc/system/default/server.conf, but not exactly user. Anyway, I edited the min minFreeSpace = 500, but modifying the setting shows following warning in the dashbaord:
Installed Files Integrity Checker: Unable to access or parse the contents of manifest file in SPLUNK_HOME directory. As a result, file integrity information is not available. Verify manifest file in SPLUNK_HOME directory is still present, and that the splunk service user context will have read-access.

It would be nice if we can have SII to configure with lower limit than 200G than above min free space settings.