Getting Data In

How limit my index growth

Communicator

Hi Splunk Users,

I am having an issue with my indexes growing very large and clogging up the space on my disk.

For example: I have noticed the index 'perfmon' getting very large so I went ahead and set the limit to 5 GB. I was reading once the limit is reached it would clean up automically and delete older data. However I see in Fire Brigade that the index size is still 25 GB. How can that be if I limited to be 5 GB?

Thank you,

Oliver

Tags (3)
0 Karma
1 Solution

SplunkTrust
SplunkTrust

Hi omuelle1, to clarify, I am assuming that you set "maxTotalDataSizeMB" for the index to 5000. If that is the case, some possible explanations:

-Splunk hasn't been restarted and needs to be in order for the change to take effect
-There is something in the way as far as file permissions go, and splunk can't delete the buckets. Check splunkd.log
-Fire Brigade (haven't worked with that) is reporting false information, or old information. do a " du -sh /path/to/index " to find out the current size

Let me know if any of this helps!

View solution in original post

SplunkTrust
SplunkTrust

Hi omuelle1, to clarify, I am assuming that you set "maxTotalDataSizeMB" for the index to 5000. If that is the case, some possible explanations:

-Splunk hasn't been restarted and needs to be in order for the change to take effect
-There is something in the way as far as file permissions go, and splunk can't delete the buckets. Check splunkd.log
-Fire Brigade (haven't worked with that) is reporting false information, or old information. do a " du -sh /path/to/index " to find out the current size

Let me know if any of this helps!

View solution in original post

Communicator

Thank you for the fast answer.

Yes you are right, that's the setting I used.
- I did restart Splunk
- Permissions are fine
- I did check the actual sizes in the folders and it matches what I see in FB

I went ahead and set the retention policy to 5 days
frozenTimePeriodInSecs = 432000

since I really don't need the data longer than 5 days and it actually cleared up space. I might have to clarify that the the index was already 25 GB when I set it to 5 GB max, however I was expecting that it would automatically clean it up to 5 GB.

0 Karma

SplunkTrust
SplunkTrust

Hi omulle1, glad to help! 😄

Yes, there is an interval ( I think 60 seconds by default ) that splunk will examine it's indexes and freeze buckets ( individual folders within an index directory ) based on the configuration in indexes.conf.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!