topic How limit my index growth in Getting Data In

How limit my index growth

omuelle1 — Tue, 27 Oct 2015 12:11:00 GMT

Hi Splunk Users,

I am having an issue with my indexes growing very large and clogging up the space on my disk.

For example: I have noticed the index 'perfmon' getting very large so I went ahead and set the limit to 5 GB. I was reading once the limit is reached it would clean up automically and delete older data. However I see in Fire Brigade that the index size is still 25 GB. How can that be if I limited to be 5 GB?

Thank you,

Oliver

Re: How limit my index growth

muebel — Tue, 27 Oct 2015 12:37:51 GMT

Hi omuelle1, to clarify, I am assuming that you set "maxTotalDataSizeMB" for the index to 5000. If that is the case, some possible explanations:

-Splunk hasn't been restarted and needs to be in order for the change to take effect
-There is something in the way as far as file permissions go, and splunk can't delete the buckets. Check splunkd.log
-Fire Brigade (haven't worked with that) is reporting false information, or old information. do a " du -sh /path/to/index " to find out the current size

Let me know if any of this helps!

Re: How limit my index growth

omuelle1 — Tue, 27 Oct 2015 13:01:35 GMT

Thank you for the fast answer.

Yes you are right, that's the setting I used.
- I did restart Splunk
- Permissions are fine
- I did check the actual sizes in the folders and it matches what I see in FB

I went ahead and set the retention policy to 5 days
frozenTimePeriodInSecs = 432000

since I really don't need the data longer than 5 days and it actually cleared up space. I might have to clarify that the the index was already 25 GB when I set it to 5 GB max, however I was expecting that it would automatically clean it up to 5 GB.

Re: How limit my index growth

muebel — Tue, 27 Oct 2015 13:14:12 GMT

Hi omulle1, glad to help! 😄

Yes, there is an interval ( I think 60 seconds by default ) that splunk will examine it's indexes and freeze buckets ( individual folders within an index directory ) based on the configuration in indexes.conf.