Does KO explorer show which fields are indexed and which not? This has always been a challenge and anything which does this would be helpful.
I couldn't find a direct answer in the doc or screenshots, but may have just missed it.
This is handy to know if you want to manipulate lispy etc. and do other optimisations on searches.
No, the KO Explorer doesn't distinguish between indexed fields and search time fields. It translates your search into normalizedSearch, indexed fields don't play a role there - that's lispy (see https://conf.splunk.com/watch/conf-online.html?search=fn1003 for more).
From a search perspective it's not easy to answer "is this field indexed or not?", at least not generically. In principle the answer could be different from one event to the next. Things you can do:
| tstats count where index=foo sourcetype=bar by maybe_indexed_fieldwill only return results if maybeindexedfield is indexed. Similarly, if
index=foo sourcetype=bar field=valueyields results but
index=foo sourcetype=bar field::valuedoes not,
fieldis likely not indexed [caveat: some weirdnesses exist with :: around complex values such as spaces, quotes, etc.]
| walklex index=_internal type=field
Finally, you can promote https://ideas.splunk.com/ideas/E-I-70 ("Add an Explain/Debug Mode to display how fields were created") to maybe get an improvement added to core in some future release.
I have voted for the Idea as suggested.
But there's still something here I don't get. I thought indexed fields DID play a role in the lispy. If you happen to know you are dealing with one, you can use the :: operator instead of =, and the lispy is different.
But since there is no easy way to figure out whether a field is indexed or not (for custom fields) you don't have this option.
Perhaps a future version of KO Explorer could at least inspect fields.conf and let the user know if a field might be indexed? Or is this not worth doing?
I am finding lispy quite confusing. I am a splunker of long standing but haven't looked under this particular rock before now.
Indexed fields can play a role in lispy, but the KO Explorer doesn't deal with lispy. Its purpose is knowledge objects, ie search time things. Tags, Eventtypes, calculated fields, field aliases, stuff like that... things that cause normalizedSearch to explode in bad cases. Indexed fields don't matter there.
A key word in that first sentence is can. Many indexed fields will work even if lispy treats them as search time, ie if
field=value is translated to a lispy of
[ AND value ] instead of
[ AND field::value ] (see linked conf talk for more depth and breadth). As a result, fields.conf will only yield a small fraction of indexed fields.
I've added two more things to consider to the original answer.
Accepted the answer, which is of course correct but still leaves open the substance of your Idea.
I had forgotten about INDEXED_EXTRACTIONS. Of course fields.conf won't give you anywhere near complete coverage. You would potentially need to go out to all the forwarders and see what they're up to as well. Alternatively, this is a good use case for splunking your git repo of splunk configurations I would think.
But in general it seems, no magic bullets for this. C'est la vie..