Getting Data In

How limit my index growth

omuelle1
Communicator

Hi Splunk Users,

I am having an issue with my indexes growing very large and clogging up the space on my disk.

For example: I have noticed the index 'perfmon' getting very large so I went ahead and set the limit to 5 GB. I was reading once the limit is reached it would clean up automically and delete older data. However I see in Fire Brigade that the index size is still 25 GB. How can that be if I limited to be 5 GB?

Thank you,

Oliver

Tags (3)
0 Karma
1 Solution

muebel
SplunkTrust
SplunkTrust

Hi omuelle1, to clarify, I am assuming that you set "maxTotalDataSizeMB" for the index to 5000. If that is the case, some possible explanations:

-Splunk hasn't been restarted and needs to be in order for the change to take effect
-There is something in the way as far as file permissions go, and splunk can't delete the buckets. Check splunkd.log
-Fire Brigade (haven't worked with that) is reporting false information, or old information. do a " du -sh /path/to/index " to find out the current size

Let me know if any of this helps!

View solution in original post

muebel
SplunkTrust
SplunkTrust

Hi omuelle1, to clarify, I am assuming that you set "maxTotalDataSizeMB" for the index to 5000. If that is the case, some possible explanations:

-Splunk hasn't been restarted and needs to be in order for the change to take effect
-There is something in the way as far as file permissions go, and splunk can't delete the buckets. Check splunkd.log
-Fire Brigade (haven't worked with that) is reporting false information, or old information. do a " du -sh /path/to/index " to find out the current size

Let me know if any of this helps!

omuelle1
Communicator

Thank you for the fast answer.

Yes you are right, that's the setting I used.
- I did restart Splunk
- Permissions are fine
- I did check the actual sizes in the folders and it matches what I see in FB

I went ahead and set the retention policy to 5 days
frozenTimePeriodInSecs = 432000

since I really don't need the data longer than 5 days and it actually cleared up space. I might have to clarify that the the index was already 25 GB when I set it to 5 GB max, however I was expecting that it would automatically clean it up to 5 GB.

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi omulle1, glad to help! 😄

Yes, there is an interval ( I think 60 seconds by default ) that splunk will examine it's indexes and freeze buckets ( individual folders within an index directory ) based on the configuration in indexes.conf.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...