topic Re: how can i list all indexes and sourcetypes?! in Splunk Search

how can i list all indexes and sourcetypes?!

r999 — Wed, 19 Dec 2012 11:36:43 GMT

i can do

| metadata type=sourcetypes |table sourcetype

but what i would like is the equivalent of:

| metadata type=sourcetypes index=* | table index sourcetype

however this does not work and does not enter data in the index column

How can i achieve this very simple list, preferably without using stats command

Re: how can i list all indexes and sourcetypes?!

martin_mueller — Wed, 19 Dec 2012 12:12:04 GMT

You can get a list of indexes like this:

 | eventcount summarize=f index=* index=_* | dedup index | fields index

See http://splunk-base.splunk.com/answers/39370/is-it-possibl-to-get-a-list-of-available-indices

Your search doesn't work because metadata does not contain any field "index". It does give you the list of sourcetypes though.

Re: how can i list all indexes and sourcetypes?!

esix_splunk — Fri, 25 Dec 2015 05:22:42 GMT

I think these solutions are overkill, and perhaps less efficient. Let's use tstats and go home early.. (its not the stats command.. 😛 )

| tstats values(sourcetype) where index=* group by index

Re: how can i list all indexes and sourcetypes?!

tmerry — Tue, 12 Jan 2016 23:15:50 GMT

I generally would prefer to use tstats (and am trying to get better with it!), but your string does not return all indexes and sourcetypes active in my environment. When I use this tstats search:

| tstats values(sourcetype) as sourcetype where index=* OR index=_* group by index

I get 19 indexes and 50 sourcetypes.

When i use the accepted answer (eventcount) i get 30 indexes and 295 sourcetypes.

I tried excluding index=_* from both searches and still saw a huge difference in the results. Any thoughts on why there is a discrepancy?

Re: how can i list all indexes and sourcetypes?!

esix_splunk — Wed, 13 Jan 2016 01:06:33 GMT

index=*

Make sure you use that and not just index=, especially if you have search filters setup so that not all indexes are searched by default.

Regarding excluding index=_*, these are internal indexes for Splunk. Of course if you are skipping these and expecting them to be in the event count, then your numbers will be off.

Re: how can i list all indexes and sourcetypes?!

tmerry — Thu, 14 Jan 2016 21:09:56 GMT

Sorry, the asterisks were stripped out of my comment, but they were there when I did my comparison.

| tstats values(sourcetype) as sourcetype where index=* OR index=_* group by index

I added the internal indexes to your proposed tstats search to match the search string in the accepted answer above. If I remove them from both searches, I still see a major discrepancy in results.

Re: how can i list all indexes and sourcetypes?!

Dallastek — Thu, 14 Jan 2016 22:48:42 GMT

I get a list of all of them

Re: how can i list all indexes and sourcetypes?!

mrjoshua050 — Thu, 21 Jul 2016 14:58:01 GMT

This was a perfect answer exactly what I needed, and very fast.

Re: how can i list all indexes and sourcetypes?!

furby559 — Thu, 21 Jul 2016 21:06:28 GMT

I downvoted this post because need to run this over all time for this to be accurate and is then significantly slower over larger data sets.

Re: how can i list all indexes and sourcetypes?!

jagadeeshm — Thu, 26 Jan 2017 01:45:06 GMT

so what did you end-up doing?

Re: how can i list all indexes and sourcetypes?!

SarahBOA — Thu, 26 Jan 2017 15:55:59 GMT

We used tstats and we only run it on part of the data. We really wanted a list of which hosts send what sourcetype and source to what index. We run it on a small sampling of the data and collect it weekly and add it to our own lookup/csv to keep track.

Re: how can i list all indexes and sourcetypes?!

jonuwz — Mon, 20 Feb 2017 18:05:24 GMT

Does this involve any setup ? the docs indicate that you need to run tscollect to create the tsidx files that tstats uses. If my answer is out-dated, i'll remove it.

Re: how can i list all indexes and sourcetypes?!

martin_mueller — Mon, 20 Feb 2017 18:18:17 GMT

Our eventcount answers still are valid, though tstats can answer the same questions nowadays - no setup needed for indexed fields like sourcetype and index.

Re: how can i list all indexes and sourcetypes?!

jagadeeshm — Thu, 31 Aug 2017 11:28:35 GMT

The discrepancy is due to the fact that tstats takes selected time period into consideration. So unless you select ALL TIME, you won't be seeing all indexes and sourcetypes.

Re: how can i list all indexes and sourcetypes?!

moh30ka — Tue, 16 Apr 2019 18:35:36 GMT

| tstats count WHERE index=* by index sourcetype | stats values(sourcetype) by index,| tstats count WHERE index=* by index sourcetype | stats values(sourcetype) by index

Re: how can i list all indexes and sourcetypes?!

sjbriggs — Thu, 10 Oct 2019 17:19:54 GMT

For some reason, I get fewer results with tstats recommendation than I get with the first recommendation. I have one index that has 3 sourcetypes and with tstats, it only shows one of them.

Re: how can i list all indexes and sourcetypes?!

chutuo — Wed, 29 Jan 2020 17:39:05 GMT

Answer by esix [Splunk] should have been the selected answed and is actually best practice!

Re: how can i list all indexes and sourcetypes?!

hardik_d — Tue, 15 Jun 2021 09:56:31 GMT

with group not work for me:

used this:

| tstats values(sourcetype) where index=* by index

you can also try this one to get indexes based on provide sourcetype in query::

| tstats values(sourcetype) where index=* sourcetype="abc" OR sourcetype="xyz" by index

Re: how can i list all indexes and sourcetypes?!

Micheal_S — Fri, 18 Jun 2021 19:51:22 GMT

This search worked better for me over the other tstats searches. The tstats search would generate errors over too long a time.

Re: how can i list all indexes and sourcetypes?!

Micheal_S — Mon, 21 Jun 2021 14:03:48 GMT

To expand on this. I had an issue where if I did this empty indexes wouldn't show in my results. I used the following to work around that.