Why is a Splunk search throwing errors on a lookup that isn't being called?

[gwalford]

One of my users has a lookup table that they have saved appropriately into their app.

It was running just fine. Now, after the weekend, when you search for anything inside the app, Splunk throws errors about the lookup file, even if you are not calling it.

For example, if you just search for:
index=main

You will get a whole host of errors from the indexers prior to the results that look like this:

[Indexer01] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer02] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer03] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer04] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer05] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer06] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer07] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer08] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer09] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer10] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer11] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer12] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.
[Indexer13] Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'network:firewall' and lookup table 'Network_Hosts_List'.

And so on...........

Why is Splunk throwing errors on a Lookup table that isn't being referenced?

[Masa]

Possibly auto-lookup for Network_Hosts_List is enabled for the app you're running the search, while the app/add-on for the lookup is not shared.
Please check and try global permission for the lookup, auto-lookup and the app/add-on and see if the warn message goes away.

[PopcornBob]

Just went through something similar. A useful app is the lookup editor on Splunkbase. It can identify duplicate lookups in different locations. For example: I moved to using an app for my CIDR lookups, and forgot to remove the local subnets.csv from /etc/apps/search/lookups/ on my search head

[martin_mueller · Accepted solution]

There's a mismatch between an automatic lookup defined for sourcetype=network:firewall and the lookup definition Network_Hosts_List, the automatic lookup is trying to use fields that don't exist in the lookup definition.

Possible reasons include someone changed the "schema" of the lookup file by removing or renaming an essential column, or the entire file could be empty.

These messages are generated because Splunk is preparing its configuration for the search - who knows, an event in index=main might have sourcetype=network:firewall, so Splunk's complaining ahead of time "this isn't going to end well" when the config mismatch was detected.

[martin_mueller]

Make sure the set of fields in the lookup file lines up with the automatic lookup, and make sure the lookup file and definition are shared at least as wide as the automatic lookup.

[gwalford]

The lookup file is intact, it isn't missing any fields.

If I call it like this:
| inputlookup Network_Hosts_List

It returns the .csv properly with no errors.

If I perform any search inside the customers app (except the above) on any index (even internal indexes) the errors pop up in the search. Even if I make sure to select an index that has no events of sourcetype=network:firewall, these errors still pop up.

[sundareshr]

There could be an automatic lookup.