How do I resolve a warning about incomplete metadata results (after 100000+ entries)?

[pavanae]

How to resolve the warning "Metadata results may be incomplete: 100000 entries have been received from all peers , and this search will not return metadata information for any more entries."

I have a query as follows

PART:-1

| inputlookup ABCD

| search Forward="Yes"
| table Region,IPHost, ip_address

| rename Region AS my_region, IPHost AS my_hostname, ip_address AS my_ip

PART:-2

| join type=left my_hostname
[| metadata type=hosts index=*
| rename host AS my_hostname]
| eval lastTime=coalesce(lastTime,0)
| eval timeDiff=now()-lastTime
| eval last_seen_in_24_hours=if(timeDiff>86400,"NO","YES")
| eval lastReported=if(lastTime=0,"never",strftime(lastTime,"%F %T"))
| table my_region,my_hostname,last_seen_in_24_hours,lastReported

where part 1 is just a csv file which displays a bunch of hosts and part is the query which checks all those bunch of hosts were actually reported in last_24_hours or not which displays the result as follows

From the above you can see that all the 3 hosts were reporting as "NO" in the dashboard which means that they were not reported in last 24 hours but all those 3 hosts were actually reporting.

To investigate further I tried to check if the part 2 works for a single host which shows as not reporting to splunk though its reporting as below

| metadata type=hosts index=* | search host="abcd"

result :- no results found

warning :- Metadata results may be incomplete: 100000 entries have been received from all peers (see parameter maxcount under the [metadata] stanza in limits.conf), and this search will not return metadata information for any more entries.

Is there any way that i can filter the search or exclude all those results. I think the above warning causing the issue and displaying all the hosts as "NO" though they were reporting insted of "YES".

Is there any other way other than modifying the limits.conf I can filter or modify my search to look only for the csv file hosts instead of looking through all the entries.

Any suggestions would be really helpful.

[DalJeanis]

Try this ...

| metadata type=hosts index=* 
| rename host AS my_hostname
| append 
    [| inputlookup ABCD 
     | search Forward="Yes" 
     | table Region,IPHost, ip_address 
     | rename Region AS my_region, IPHost AS my_hostname, ip_address AS my_ip
     ]
| stats values(my_region) as my_region, values(my_ip) as my_ip, max(lastTime) as lastTime by my_hostname 
| eval lastTime=coalesce(lastTime,0)
| eval timeDiff=now()-lastTime
| eval last_seen_in_24_hours=if(timeDiff>86400,"NO","YES")
| eval lastReported=if(lastTime=0,"never",strftime(lastTime,"%F %T"))
| table my_region,my_hostname,last_seen_in_24_hours,lastReported

[pavanae]

Thanks for the response @DalJeanis. It's been displaying the results for all for the hosts but I'm trying to get the results only the hosts in lookup(ABCD).

[pavanae]

@DalJeanis could you please remodify the query to display the hosts from the lookup(ABCD). The above query results 1000's of hosts but the csv file contains 600 hosts.

[somesoni2]

Just replace first two lines with this

| metadata type=hosts index=* 
| search [| inputlookup ABCD | search Forward="Yes"  | table IPHost | rename IPHost as host]
 | rename host AS my_hostname
....rest of the search..