topic Re: Index viewable in Events but not in Statistics in Splunk Search

Index viewable in Events but not in Statistics

Lucas_Henry_ — Wed, 28 Sep 2016 15:38:38 GMT

I can see events from two indexes in the Events section, but my Statistics shows only events from one of the indexes. Are there common issues that can cause this?

Re: Index viewable in Events but not in Statistics

PPape — Wed, 28 Sep 2016 15:44:46 GMT

Can you show the search you are running?

Re: Index viewable in Events but not in Statistics

Lucas_Henry_ — Tue, 29 Sep 2020 11:13:53 GMT

index=sterling | eval host=lower(host) | eval permission_type=lower(permission_type) | stats earliest(_time) as earliest latest(_time) as latest values(source) as sources by host index sourcetype splunk_server permission_type fillnull | convert ctime(earliest) ctime(latest) | table index host sourcetype earliest latest sources splunk_server permission_type | sort host

Re: Index viewable in Events but not in Statistics

Lucas_Henry_ — Tue, 29 Sep 2020 11:13:58 GMT

The indices are "sterling" and "sterling_nonprod". Sterling_nonprod is the only index that works with this search

Re: Index viewable in Events but not in Statistics

PPape — Tue, 29 Sep 2020 11:11:02 GMT

First of all only to get it right...
you run this search two times? Once with index=sterling once with index=sterling_nonprod right?
(only if not try "index=sterling OR index=sterling_nonprod")

Second is the attribute permission_type available in both of your indexed datasets (indexes)?

Re: Index viewable in Events but not in Statistics

sideview — Tue, 29 Sep 2020 11:14:03 GMT

It sounds like your search is making some assumptions about the fields present, and somehow those assumptions are only true for the one index.

in stats .... by host index sourcetype splunk_server permission_type fillnull

Obviously host, index, sourcetype and splunk_server will always be defined on all the incoming event rows. However i the extraction for permission_type isn't working right for the data in the other index or if the (very strange looking) "fillnull" field isn't really a field, then any rows with null values for either field will get thrown away. If none of the rows contain any value for permission_type or "fillnull" then you'll get no result rows out of that stats command.

I would just look at the field values directly index=sterling | stats count by fillnull or even index=sterling | table fillnull permission_type. If there are null values anywhere, that extraction or that lookup field or what-have-you, will be your culprit.