Help identifying fast growing indexes

[jwpoore]

Hi fellow Splunkers.

I am the Splunk admin at my org, however that is mainly more from the Infrastructure side of things so when it comes to actually using Splunk I am a novice. I would like to change this but one thing at a time, Splunk is only one of my problems ;).

We've got 4 Indexers, 2 in each DC. Up until last week these there pretty consistent with each other in terms of growth although now one site is growing about 30GB per day quicker than the other. This isn't a big deal, but I'd like to know why.

Can someone help me with a search which shows growth per day vs the previous day? Or have any tips to help me try and narrow down what's actually growing faster than normal.

Appreciate any help you can offer.

[MattibergB · Accepted solution]

Hi,

You could use something like the following:

  index=_internal earliest=-2d@d latest=-0d@d
    source=/opt/splunk/var/log/splunk/license_usage.log* 
| eval gb=round(b/1024/1024/1024,2)
|bin span=1d _time
| stats sum(gb) as gb by idx _time
| sort by idx

it will show the gb per index per day, you can change the earliest to find older data.
You could add splunk_server to stats sum(gb) as gb by idx _time splunk_server
Then you can see the difference between your servers.

[adonio]

there are tons of answers on this portal, here are a couple:
https://answers.splunk.com/answers/716733/how-do-you-calculate-the-growth-of-each-index-on-a.html
https://answers.splunk.com/answers/173623/how-to-get-size-counters-for-splunk-indexes-over-a.html

and like @richgalloway mentioned, use the monitoring console

[richgalloway]

I believe the Monitoring Console has dashboards that show index growth over time.