topic Re: Is there a search for indexes/sources that aren't being used... in Splunk Search

Is there a search for indexes/sources that aren't being used...

a212830 — Mon, 14 Dec 2015 15:09:00 GMT

Is there a way to determine what sources and/or sourcetypes AREN'T being searched? If data is coming into Splunk and nobody is really looking at that data, then I don't need to keep bringing it in. I just want to find a way to determine this.

Re: Is there a search for indexes/sources that aren't being used...

javiergn — Mon, 14 Dec 2015 15:46:46 GMT

This is what I would do for sourcetypes:

index=_audit action=search user=* search=* "sourcetype"
| fields user, search
| rex field=search max_match=0 "sourcetype\s*=\s*(?<st_used>[\w\d_]+)"
| stats count by user, st_used
| sort limit=0 st_used

You can probably use something similar for indexes or sources.

Thanks,
J

Re: Is there a search for indexes/sources that aren't being used...

yannK — Mon, 14 Dec 2015 17:16:57 GMT

It will not really show you all the data that are being searches implicitly, but will give you an idea of what is searched the most explicitly.

explicit searches :

sourcetype=A
NOT sourcetype=B
implicit searches :

sourcetype=*
index=B *

Re: Is there a search for indexes/sources that aren't being used...

NullZero — Wed, 03 Nov 2021 12:00:33 GMT

Regarding indexes which are not being used, a simple adaptation of the example already posted yields a count of searches by index. I did this over 30 days as a quick example:

index=_audit action=search user!=splunk-system-user search=* "index"
| rex "search index=\"(?<unused_index_search>\w+)"
| stats count by unused_index_search
| sort - count

This can then be used to compare to a list of indexes and simply lookup count. An extension could be to use a lookup in conjunction but this is simple.

As mentioned already, always check with the client and end users before removing anything.