topic Re: Query HDD space, index and data in Splunk Search

Query HDD space, index and data

rene847 — Mon, 28 Sep 2020 20:16:54 GMT

Hi,
I have not been able to find a good query with all my trying.... I need help please!

Can anyone tell how I can:
I would like to get a query that would give me the disk space since December 1, 2014 for each month until today (to see the progression).
Splunk has sent me alert each day (email) about the status of our indexes (view and control our licenses). I use the same alert with these values "-198d@d to now" and it doesn't work, I only view 1 month of data and I don't know why.

I'm looking for the same query (or similar) for:
- all space
- all the indexes
- all data entries
==> number of entry since December 1, 2014 until today for each month.

Is it possible? Do you have an query idea?

Thank you
Regards

Re: Query HDD space, index and data

martin_mueller — Wed, 17 Jun 2015 19:27:31 GMT

Did you take a look at the reports included in the Distributed Management Console?

Re: Query HDD space, index and data

rene847 — Thu, 18 Jun 2015 18:32:40 GMT

Yes, that way as possible and I tried.
However, I can not take a value greater than 30 days otherwise I have no data?
Do you have an idea?

Re: Query HDD space, index and data

martin_mueller — Thu, 18 Jun 2015 19:29:36 GMT

As per default settings, Splunk only retains thirty days of data in _internal. You have two options to change that for the future.

The easy way out: Increase the retention time for the index. You'll need a lot more disk space, but it's a simple change and you'll have all the data available.

The efficient way: Set up summary indexing that for example runs daily and grabs yesterday's data, calculates a daily report and stores that in a summary index with long retention time. When you want to run an overall report you simply run it on the pre-aggregated data in the summary index for a fast result with minimal space used. It's a little more effort than the easy way out, and you can't add more data to your summary retroactively just like you can't retroactively increase the retention time and regain old data that was deleted...

Re: Query HDD space, index and data

MuS — Thu, 18 Jun 2015 19:40:08 GMT

Hi rene847,

all Splunk internal indexes like _internal and _introspection have a default retention of 30 days. You can check it like this:

splunk cmd btool indexes list --debug _internal | grep frozenTimePeriodInSecs
/opt/splunk/etc/system/default/indexes.conf frozenTimePeriodInSecs = 2592000

Change it in indexes.conf but be aware of the increasing disk space needs. Also you may need to adapt the maxTotalDataSizeMB option which is by default at 500000.

Hope that helps ...

cheers, MuS

Re: Query HDD space, index and data

MuS — Thu, 18 Jun 2015 19:40:58 GMT

.... I'm too slow in the morning 🙂

Re: Query HDD space, index and data

martin_mueller — Thu, 18 Jun 2015 19:42:42 GMT

Your mornings are all wrong 😛

Re: Query HDD space, index and data

MuS — Thu, 18 Jun 2015 19:48:03 GMT

No, not at all - I'm in a time machine. Currently I'm writing from your future 🙂

Re: Query HDD space, index and data

rene847 — Fri, 19 Jun 2015 18:22:54 GMT

Thank you Martin_Mueller and MuS for your answer, I appreciate your support.