topic Re: How to check if an index exists efficiently? in Splunk Search

How to check if an index exists efficiently?

dapitis — Tue, 12 Mar 2019 10:28:47 GMT

In elasticsearch one would do HEAD [index_name] and check if an index exists efficiently. Is it possible to do something similar in Splunk?

Re: How to check if an index exists efficiently?

lakshman239 — Tue, 12 Mar 2019 12:10:16 GMT

You could run a rest search |rest /services/data/indexes | fields title | rename title AS index

Re: How to check if an index exists efficiently?

dapitis — Tue, 12 Mar 2019 13:33:41 GMT

This will return me all the indices and not a kind of a binary response if a specific index exists or not (for the elasticsearch case one could check the status code of the response).

A possible solution seems to be this one but I am not sure how efficient it is:
| eventcount summarize=false index=[index_name] | dedup index | fields index

Re: How to check if an index exists efficiently?

lakshman239 — Tue, 12 Mar 2019 14:40:51 GMT

you could just add |rest /services/data/indexes | fields title | rename title AS index | search index=yourindexname

Re: How to check if an index exists efficiently?

gjanders — Tue, 12 Mar 2019 22:06:07 GMT

I have used:

| eventcount summarize=false index=* index=_*

And found it to be very efficient, it does not require the ability to run REST queries on the indexers so 1 less capability the user needs.

Re: How to check if an index exists efficiently?

jtrujillo — Wed, 13 Mar 2019 02:29:35 GMT

You can also just head to the GUI (flash timeline search) to use the typeahead feature for dummyproof results. (e.g. type index=p and you get all the indexes that start with a p in which your particular RBAC has access for)

BUT CHECK THIS OUT! ... That doc I linked above shows typeahead being used as a search/api/ish feature that you can utilize for fast recovery of knowledge objects! TIL.

| typeahead prefix=source count=10 index=_internal

Re: How to check if an index exists efficiently?

MuS — Wed, 13 Mar 2019 02:50:51 GMT

Hi dapitis,

the most efficient way in Splunk is to use tstats like this :

 | tstats count WHERE index=<PlaceYourIndexNamehere>

This will not search any _raw data and therefore it is very, very fast 😉
Don't forget to set the correct time range if you run the search.

hope that helps ...

cheers, MuS

Re: How to check if an index exists efficiently?

gjanders — Wed, 13 Mar 2019 11:43:40 GMT

While tstats is much faster than a normal search (doing index=... | stats), and tstats is time aware which may suit your needs. However that said, tstats is not as fast as the eventcount command or the REST queries that you could be able to use.

As per my post I prefer eventcount if you just want a list of available indexes, keep in mind the eventcount command is not time aware as per the documentation:

Specifying a time range has no effect on the results returned by the eventcount command. All of the events on the indexes you specify are counted.

If the goal is to find information that is time-based then use tstats as per MuS's post.

Re: How to check if an index exists efficiently?

DMohn — Wed, 13 Mar 2019 12:03:19 GMT

Be aware that all tstats or eventcount or similar searches will return a result even if the index is not created at all. The only way to really tell if an index is there would be calling the REST interface.

| rest /services/data/indexes | stats count(eval(title=="yourindex")) as exists

This search will return 0 if your index does not exist, and 1 if it exists - even if it has no events.

If you save this search as a eval macro and pass the index name as a variable you can include this in any search if needed.

Re: How to check if an index exists efficiently?

DMohn — Wed, 13 Mar 2019 12:04:10 GMT

Be aware that this search will return a valid result even if the index is not existant at all. I guess this is not what was asked.

Re: How to check if an index exists efficiently?

MuS — Wed, 13 Mar 2019 19:03:47 GMT

Not sure what version you are running but on Splunk 7.1.4 this SPL

| tstats count WHERE index=DieserIndexExistiertNicht by index

returns No results found.
Also this one

| tstats count WHERE index=DieserIndexExistiertNicht

returns a count of 0

One might claim this a valid result ? IMHO it is not

Re: How to check if an index exists efficiently?

MuS — Wed, 13 Mar 2019 19:17:47 GMT

And yet another be aware post 😉

This will only work if you either:

run it on the indexer itself and it will only return the indexes this indexer has configured
run it on an instance that has indexers configured as search peer and if you have the correct permissions granted then it will show all configured indexes - otherwise you will get this error Restricting results of the "rest" operator to the local instance because you do not have the "dispatch_rest_to_indexers" capability.

All of the SPL commands listed in all the posts are valid and useful, all of them have their limitations and all of them can be used to achieve the same, if you keep all the be awares in mind 😉

cheers, MuS

Re: How to check if an index exists efficiently?

DMohn — Wed, 13 Mar 2019 20:10:33 GMT

An yes, of course you are right 🙂

The famous world of Splunk, with all its culprits and pitfalls...

Re: How to check if an index exists efficiently?

Esky73 — Thu, 19 Mar 2020 23:16:53 GMT

... and doesn't work on metric indexes

but thx - worked for event idx's