Splunk Search

how can i list all indexes and sourcetypes?!

Path Finder

i can do

| metadata type=sourcetypes |table sourcetype

but what i would like is the equivalent of:

| metadata type=sourcetypes index=* | table index sourcetype

however this does not work and does not enter data in the index column

How can i achieve this very simple list, preferably without using stats command

New Member

| tstats count WHERE index=* by index sourcetype | stats values(sourcetype) by index,| tstats count WHERE index=* by index sourcetype | stats values(sourcetype) by index

0 Karma

Explorer

When I have tried using the above tstats I don't get all of my indexes/sourcetypes. When I use | eventcount summarize=false index=* index=_* | dedup index | fields index | map maxsearches=100 search=" | metadata type=sourcetypes index=\"$index$\" | eval index=\"$index$\"" | stats values(sourcetype) by index

I get a list of all of them

0 Karma

Splunk Employee
Splunk Employee

I think these solutions are overkill, and perhaps less efficient. Let's use tstats and go home early.. (its not the stats command.. 😛 )

| tstats values(sourcetype) where index=* group by index

Engager

Answer by esix [Splunk] should have been the selected answed and is actually best practice!

0 Karma

Explorer

For some reason, I get fewer results with tstats recommendation than I get with the first recommendation. I have one index that has 3 sourcetypes and with tstats, it only shows one of them.

0 Karma

Influencer

Does this involve any setup ? the docs indicate that you need to run tscollect to create the tsidx files that tstats uses. If my answer is out-dated, i'll remove it.

0 Karma

SplunkTrust
SplunkTrust

Our eventcount answers still are valid, though tstats can answer the same questions nowadays - no setup needed for indexed fields like sourcetype and index.

New Member

I downvoted this post because need to run this over all time for this to be accurate and is then significantly slower over larger data sets.

0 Karma

Contributor

so what did you end-up doing?

0 Karma

Path Finder

We used tstats and we only run it on part of the data. We really wanted a list of which hosts send what sourcetype and source to what index. We run it on a small sampling of the data and collect it weekly and add it to our own lookup/csv to keep track.

0 Karma

New Member

This was a perfect answer exactly what I needed, and very fast.

0 Karma

Explorer

I generally would prefer to use tstats (and am trying to get better with it!), but your string does not return all indexes and sourcetypes active in my environment. When I use this tstats search:

| tstats values(sourcetype) as sourcetype where index=* OR index=_* group by index

I get 19 indexes and 50 sourcetypes.

When i use the accepted answer (eventcount) i get 30 indexes and 295 sourcetypes.

I tried excluding index=_* from both searches and still saw a huge difference in the results. Any thoughts on why there is a discrepancy?

Splunk Employee
Splunk Employee
index=*

Make sure you use that and not just index=, especially if you have search filters setup so that not all indexes are searched by default.

Regarding excluding index=_*, these are internal indexes for Splunk. Of course if you are skipping these and expecting them to be in the event count, then your numbers will be off.

Explorer

Sorry, the asterisks were stripped out of my comment, but they were there when I did my comparison.

| tstats values(sourcetype) as sourcetype where index=* OR index=_* group by index

I added the internal indexes to your proposed tstats search to match the search string in the accepted answer above. If I remove them from both searches, I still see a major discrepancy in results.

0 Karma

Contributor

The discrepancy is due to the fact that tstats takes selected time period into consideration. So unless you select ALL TIME, you won't be seeing all indexes and sourcetypes.

SplunkTrust
SplunkTrust

You can get a list of indexes like this:

 | eventcount summarize=f index=* index=_* | dedup index | fields index

See http://splunk-base.splunk.com/answers/39370/is-it-possibl-to-get-a-list-of-available-indices

Your search doesn't work because metadata does not contain any field "index". It does give you the list of sourcetypes though.

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!