Errors increasing on search where "lookup table does not exist" for multiple sources but lookups have status of disabled

[srunyon]

When running ad-hoc searches, I am getting errors that are increasing. My last search produced "20 errors occurred while the search was executing. Therefore, search results might be incomplete.". When I expand the message, it shows that each message is looking for a lookup table that does not exist. An example message is "The lookup table 'ftnt_action_lookup' does not exist. It is referenced by configuration 'fgt_traffic'". When I do a view objects for the apps and find the lookup shown in the error message, each show a status of "Disabled". I tried to enable the lookups but that did not stop the errors. I went to each app and tried to locate the "savedsearches" file to get a read on what is needed to generate a lookup. I could not find any of searches that generate the lookup files. I would like to know how to either produce the lookups or stop getting the errors.

[woodcock]

Did you upgrade recently? It used to be that if you used inputlookup SomeFIle.csv and SomeFile.csv was a file reference instead of a lookup definition, Splunk would decide "You are dumb but I know what you mean so I guess that is OK". Some recent release added code to generate warnings (or maybe it is errors?) for this instead. If this is the case, all you have to do is create a lookup definition with the same name as the file and the warning/error will go away.

[srunyon]

This is the original install of 6.3.1. In creating a lookup definition, do you mean using the lookup editor to create the lookup or a different conf file besides props.conf or transforms.conf? The only lookups that seem to work on my configuration all have a search under savedsearches that define the fields to be populated.

[woodcock]

You already created the file, now you need Settings -> Lookups -> Lookup definitions.

Are you getting an error? Perhaps your lookup is out of scope? Try making the permission on it Global.

[srunyon]

The error I am getting is because there are no lookups to find that are in the props.conf file. Using the btool command from the answer by maciep, I found the LOOKUP- line that has the fields needed to create a lookup and stop the errors. A lot of work poking around just to get searches to run.

[srunyon]

After configuring the lookups, I had one that would not go away. After checking under Settings as you suggested, I found that it was not set up as Global. After changing the setting, I no longer have any errors. Thanks for the help.

[maciep · Accepted solution]

That might mean that you have an automatic lookup configured but either the lookup table doesn't exist or you don't have access to it. Anything under Settings -> Lookups -> Automatic lookups (might want to check under the "All" app context)?

I believe those are configured in props on the back end, so you should be able to run a btool too

splunk btool props list --debug | grep  fgt_traffic

[srunyon]

There are no lookups under Automatic. I was able to find these lookups have stanzas in prop.conf and transforms.conf but there is nothing in savedsearches. My understanding is that a search has to be run to populate a file or a lookup created manually (either using the lookup editor app or creating a csv file then placing the file in the lookup folder for the app.

[maciep]

well that's one way to populate a lookup but not the only way. And your error isn't a result of failing to populate a lookup, it's trying to read from one that doesn't exist.

And after googling fgt_traffic, I'm guessing this is part of the fortinet add-on? When I downloaded that, I do see a csv file in the lookups directory named ftnt_action_info.csv. It is pre-populated with some static lookup data.

In transforms, that lookup is defined:

[ftnt_action_lookup]
filename = ftnt_action_info.csv

In props, an automatic lookup is configured:

LOOKUP-fgt_traffic_action           = ftnt_action_lookup ftnt_action OUTPUT action

So when you run a search, Splunk is trying to do that lookup, automatically behind the scenes. The config in props tells it what to lookup and in which table. The entry in transforms tells splunk the actual filename for that lookup table. Then splunk looks up the action field in that csv.

In your environment, I'm guessing either the csv doesn't exist or you don't have permissions to it or the definition. I don't have that TA installed, so I can't do much testing. i was just looking at the folder structure after downloading. You might want to start in the Splunk_TA_fortinet_fortigate app if that is what you're using. Is the csv file in lookups. Is the definition in transforms. And then check the web to see if you have rights to them? Maybe set any of them to global if needed (think they should be by default).

Hope that's relevant and can maybe point you in the right direction

[srunyon]

After running the btool as you suggested, I found the LOOKUP- line that provides the fields for the lookup. I then went to lookups/lookup definitions to find the app and the lookup file name to use. I then created a lookup file using the lookup editor app and the error is no longer there. I am having a problem with the ones that have "subtype" in the LOOKUP line.

[maciep]

not sure if I know what is meant by "subtype". Do you have an example of one of those lookup lines you can share? Are the error messages the same for those or different?

[srunyon]

This is the line is default props.conf with subtype -
LOOKUP-fgt_event_action= ftnt_event_action_lookup subtype vendor_action vendor_status OUTPUT action, change_type
The error message is -
Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'fgt_event' and lookup table 'ftnt_event_action_lookup'.

[maciep]

So it sounds like for that error, the problem is that the fields in the lookup file don't seem to match the fields being requested by that lookup command. Do you know which file that is associated?

If so, are their header fields for "subtype", "vendor_action", "vendor_status", "action" and "change_type"? If not, maybe just add them?

[srunyon]

That worked.