topic Re: Find Historical Index Volume Usage in Installation

Find Historical Index Volume Usage

sdyawg — Mon, 28 Sep 2020 14:13:21 GMT

I am operating on an old 4.3.1 instance of Splunk. Recently I've built up our infrastructure with three new Indexers/Deployment Servers and two new search heads. I am able to view the index volume usage for the past two months since I started the project, but I am unable to obtain historical data pertaining to the volume of events being indexed.

This Splunk-base question worked perfectly for the past 2 months of data, but I cannot see anything beyond that. When I search for data previous to this time, I receive "No results found". This is the search string I have been using.

index=_internal source=*metrics.log splunk_server="local" | eval MB=kb/1024 | search group="per_index_thruput" | chart sum(MB) by series | sort - sum(MB)

I believe the problem may lie in there not being any metric logs beyond that point in time, is there any way to have Splunk evaluate all indexed events from a certain time/date/range and show me how much has been indexed on those days? I am evaluating my past license usage in preparation to create a business case to present to my execs to purchase a larger license.

Re: Find Historical Index Volume Usage

linu1988 — Mon, 28 Sep 2020 14:13:24 GMT

Could you check whether your _internal index has the historical data for anything for that period?
You can also try this search to get the usage.
index=_internal source="license_usage." |eval GB=b/1024/1024/1024)|rename GB as Usage_Stats

Re: Find Historical Index Volume Usage

sdyawg — Mon, 01 Jul 2013 21:29:42 GMT

it looks like there is nothing in my _internal index for anything beyond two months ago. I have definitely used a few search strings to find answers like this before, but there's simply nothing in that index currently.

Checked the Splunk Manager: "Earliest Event: May 14, 2013 6:57:20 AM"

Is there anyway to pull index volume usage from before this time?

Re: Find Historical Index Volume Usage

tiny3001 — Wed, 18 Sep 2013 10:30:23 GMT

This has to do with the data retention policy on the _internal index.

Look at: indexes.conf

Specifically the setting for: frozenTimePeriodInSecs

If you increase that, you should be able to store larger periods in your _internal index.