Archive

How to calculate the index size from all indexers

Contributor

Is there a way to calculate total size of an index from all indexers?
I can see index size from each individual indexer but i have around 30 indexers in the setup i am working on..so ideally wouldn't work if go and check index size in each indexer individually

Tags (2)
1 Solution

SplunkTrust
SplunkTrust

Building on MuS' answer, you can do each indexer from a single search head. This will require SoS to be installed ( http://apps.splunk.com/app/748/ ).

| inputlookup splunk_servers_cache | search server_role="search-peer" | map maxsearches=100 search="| rest /services/data/indexes splunk_server=$sos_server$ " | stats sum(currentDBSizeMB) by title splunk_server

Each of these calls should be really quick, so the number of search-peers will increase execution time, but not as badly as if you were doing a data search.

You can also adjust the stats do whatever you need, | stats sum(currentDBSizeMB) by title.

View solution in original post

SplunkTrust
SplunkTrust

Building on MuS' answer, you can do each indexer from a single search head. This will require SoS to be installed ( http://apps.splunk.com/app/748/ ).

| inputlookup splunk_servers_cache | search server_role="search-peer" | map maxsearches=100 search="| rest /services/data/indexes splunk_server=$sos_server$ " | stats sum(currentDBSizeMB) by title splunk_server

Each of these calls should be really quick, so the number of search-peers will increase execution time, but not as badly as if you were doing a data search.

You can also adjust the stats do whatever you need, | stats sum(currentDBSizeMB) by title.

View solution in original post

Path Finder

SoS is discontinued and no longer supported. Is there an alternative query that can be run now?

My apologies if I should have posted this in a new question.

0 Karma

SplunkTrust
SplunkTrust
| rest /services/data/indexes |  stats sum(currentDBSizeMB) by title splunk_server

The above might do it, you can use the REST API or use the monitoring console. The monitoring console has a variety of queries that do this which you can borrow by clicking on the magnifying glass (it replaces SoS)

SplunkTrust
SplunkTrust

You can even summarize this (with |collect index=summary) and then do data capacity planning!

0 Karma

Contributor

Thanx a lot...I do have S.o.S installed..and running this query provided me the data i require

0 Karma

Influencer

The eventcount command may be what you need:

http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Eventcount

Haven't tried it against multiple indexers however.

This command requires a "|" before the command to run.

Hope this helps.

Influencer

@adityapavan18, again I'm not sure, as I only one indexer to play around with at the moment, but perhaps the following, seems odd to me that it has the server field if it is localized to one server...

| eventcount summarize=false index=* report_size=true | eval MB=(size_bytes/1024)/1024 | stats sum(MB) by index, server

Splunk Employee
Splunk Employee

worked for me - thanks MHibbin

0 Karma

Explorer

Nope doesn't work..

0 Karma

Contributor

this helps only for a particular indexer.Not sure how we use for multiple indexers

0 Karma

SplunkTrust
SplunkTrust

Hi adityapavan18

one way would be to use the REST endpoint /services/data/indexes for that, but this must be done against each indexer. Personally I would setup a summarized saved search on each indexer which runs the following search:

     | rest /services/data/indexes | stats values(currentDBSizeMB) by title

This way you will be able to get the index size for each indexer with one single search afterwards.

hope this helps ...

cheers, MuS

Contributor

this is way better, as anything that requires the 3rd party app like SOS is not a clean one IMHO. i think more and more of these weird little tasks must be phased out and worked into core splunk (via REST API for instance or DMC moving forward.

0 Karma

Contributor

Thanx MuS.

0 Karma

Explorer

| rest /services/data/indexes | stats values(currentDBSizeMB) by title

That only provides data of index per indexer, it's not added up.. How would you sum it?

0 Karma

SplunkTrust
SplunkTrust

Quick and dirty solution is to run this:

 | rest /services/data/indexes | where totalEventCount!=0 | stats values(currentDBSizeMB) AS size by title | streamstats sum(size) AS total
0 Karma

Splunk Employee
Splunk Employee

Note that currentDBsizeMB is 1 for an index with no events. Might want to filter out those where totalEventCount=0

0 Karma

SplunkTrust
SplunkTrust

Good point, I changed the search - thanks 😉

0 Karma

Path Finder

what you changed? paste it here plz

0 Karma

Communicator

why not just use | stats sum(currentDBSizeMB) by title

instead of values?

0 Karma

SplunkTrust
SplunkTrust

Because currentDBSizeMB is the size of each index and there is no need to sum() them.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!