Getting Data In

How limit my index growth

omuelle1
Communicator

Hi Splunk Users,

I am having an issue with my indexes growing very large and clogging up the space on my disk.

For example: I have noticed the index 'perfmon' getting very large so I went ahead and set the limit to 5 GB. I was reading once the limit is reached it would clean up automically and delete older data. However I see in Fire Brigade that the index size is still 25 GB. How can that be if I limited to be 5 GB?

Thank you,

Oliver

Tags (3)
0 Karma
1 Solution

muebel
SplunkTrust
SplunkTrust

Hi omuelle1, to clarify, I am assuming that you set "maxTotalDataSizeMB" for the index to 5000. If that is the case, some possible explanations:

-Splunk hasn't been restarted and needs to be in order for the change to take effect
-There is something in the way as far as file permissions go, and splunk can't delete the buckets. Check splunkd.log
-Fire Brigade (haven't worked with that) is reporting false information, or old information. do a " du -sh /path/to/index " to find out the current size

Let me know if any of this helps!

View solution in original post

muebel
SplunkTrust
SplunkTrust

Hi omuelle1, to clarify, I am assuming that you set "maxTotalDataSizeMB" for the index to 5000. If that is the case, some possible explanations:

-Splunk hasn't been restarted and needs to be in order for the change to take effect
-There is something in the way as far as file permissions go, and splunk can't delete the buckets. Check splunkd.log
-Fire Brigade (haven't worked with that) is reporting false information, or old information. do a " du -sh /path/to/index " to find out the current size

Let me know if any of this helps!

omuelle1
Communicator

Thank you for the fast answer.

Yes you are right, that's the setting I used.
- I did restart Splunk
- Permissions are fine
- I did check the actual sizes in the folders and it matches what I see in FB

I went ahead and set the retention policy to 5 days
frozenTimePeriodInSecs = 432000

since I really don't need the data longer than 5 days and it actually cleared up space. I might have to clarify that the the index was already 25 GB when I set it to 5 GB max, however I was expecting that it would automatically clean it up to 5 GB.

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi omulle1, glad to help! 😄

Yes, there is an interval ( I think 60 seconds by default ) that splunk will examine it's indexes and freeze buckets ( individual folders within an index directory ) based on the configuration in indexes.conf.

0 Karma
Get Updates on the Splunk Community!

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Enhance Security Operations with Automated Threat Analysis in the Splunk EcosystemAre you leveraging ...

Splunk Developers: Go Beyond the Dashboard with These .Conf25 Sessions

  Whether you’re building custom apps, diving into SPL2, or integrating AI and machine learning into your ...

Index This | How do you write 23 only using the number 2?

July 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...