topic Re: Viewing all Indexes and sourcetypes in use. in Splunk Search

Viewing all Indexes and sourcetypes in use.

Abraham1234 — Thu, 18 Jun 2020 16:10:21 GMT

We are in the midst of a migration from one server to the next, and need to see if there are queries running against specific indexes, virtual indexes and sourcetypes. I have been trying a number of queries against the audit log but can't find a way to extract the following information used by all active queries & reports.

1. name and count of indexes

2. name and count of virtual indexes

3. name and count of sourcetypes

Been searching for hours, any help appreciated.

Re: Viewing all Indexes and sourcetypes in use.

The_Simko — Thu, 18 Jun 2020 17:46:55 GMT

Partial Answers coming:

3. Sourcetypes
| metadata type=sourcetypes index=*.

2. Virtual Indexes
Do you have virtual indexes, as in Hadoop ones?
| rest /services/data/indexes | search isVirtual = 1

1. Indexes
| rest /services/data/indexes | search isVirtual = 0

With the rest, you can narrow your fields to find out what you are looking for.

- Michael S

Re: Viewing all Indexes and sourcetypes in use.

marycordova — Thu, 18 Jun 2020 18:24:06 GMT

https://docs.splunk.com/Documentation/SplunkCloud/8.0.2004/SearchReference/Metadata

| metadata type=[sourcetypes or sources or hosts] index=*

this will give you a list of each of the above, you might need to set your search to a broad time range, maybe at least 30 days or so depending on what you want to make sure gets migrated

also

https://docs.splunk.com/Documentation/SplunkCloud/8.0.2004/SearchReference/Tstats

| tstats values(sourcetype) WHERE index=* by index

this will give you a list of the sourcetypes by index

- upvotes appreciated 🤓

Re: Viewing all Indexes and sourcetypes in use.

gjanders — Fri, 19 Jun 2020 03:26:06 GMT

While answering the which sourcetypes/indexes are available is relatively easy, answering the question of "which of those indexes/sourcetypes were searched recently" is surprisingly difficult.

Two ideas are open on this and under consideration, in particular Better audit logs and Provide index access statistics to assist in capacity planning of the indexing tier

I put my attempts to complete this into Alerts for Splunk Admins (SplunkBase)

I also have the searches on github in particular "SearchHeadLevel - Search Queries summary exact match 73" which works in 7.3 and above, but there is definitely some complexity in getting those searches to run so you may wish to take a more simple approach...