topic Re: Why am I getting error "100000 entries have been received...this search will not return metadata information for any more entries."? in Splunk Search

Why am I getting error "100000 entries have been received...this search will not return metadata information for any more entries."?

bsellapi — Mon, 20 Jun 2016 09:02:15 GMT

I am getting below error when I use the metadata command. Could someone explain to me in detail what this is all about?

Error:

Metadata results may be incomplete: 100000 entries have been received from all peers (see parameter maxcount under the [metadata] stanza in limits.conf), and this search will not return metadata information for any more entries.

My requirement is to get the latest source for a particular index using metadata. I am using sort on the recentTime field, but I am getting above error message.

Thanks

Re: Why am I getting error "100000 entries have been received...this search will not return metadata information for any more entries."?

jmallorquin — Mon, 20 Jun 2016 13:58:12 GMT

Hi,

Which period are you using in the search? alltime?, try other if you use alltime.

Hope i help you

Re: Why am I getting error "100000 entries have been received...this search will not return metadata information for any more entries."?

bsellapi — Mon, 20 Jun 2016 14:40:19 GMT

Example

|metadata type=sources index=*-aa| search source="*test*" | sort - recentTime | rex field=source "/(?\d+/.*)\.\d+.gz" | stats first(source) as source by uniqueSource | fields source

What do you mean by "other"? Could you shed some light on that? Do you mean tstats? We want to use metadata as much as possible.

Re: Why am I getting error "100000 entries have been received...this search will not return metadata information for any more entries."?

davidpaper — Tue, 21 Jun 2016 15:13:45 GMT

Greetings,

I feel like jmallorquin is referring to using the time picker to select a time frame other than "All time" if that is what you currently have selected when running the search. You were not specific in about that in your question.

Re: Why am I getting error "100000 entries have been received...this search will not return metadata information for any more entries."?

bsellapi — Tue, 21 Jun 2016 15:19:26 GMT

I tried adding time range but still I am getting the error as above. Reason being time will be considered after the search result I believe w.r.t metadata. We will not be able to pass time rage in the input section.

Re: Why am I getting error "100000 entries have been received...this search will not return metadata information for any more entries."?

davidpaper — Tue, 21 Jun 2016 16:08:32 GMT

I notice when I run a simple search like

|metadata type=sources index=_internal

for 15 minutes vs 90 days, I get a significantly different count of results (37 vs 93 on a small test instance). What time frame are you using for this search?

Also, there is a setting in that sets the max count for metadata in limits.conf.

[metadata]
maxresultrows =

* The maximum number of results in a single chunk fetched by the metadata
command
* A smaller value will require less memory on the search head in setups with
large number of peers and many metadata results, though, setting this too
small will decrease the search performance
* Default is 10000
* Do not change unless instructed to do so by Splunk Support

maxcount =

* The total number of metadata search results returned by the search head;
after the maxcount is reached, any addtional metadata results received from
the search peers will be ignored (not returned)
* A larger number incurs additional memory usage on the search head
* Default is 100000

Note that if there are a very large number of metadata values, the memory footprint of the search might be quite large.