Why isn't my lookup working?

[willthames2]

Lookup file testing.csv looks a bit like

user,username
1234,bob
2345,jim
3456,mary

In props.conf I have

[weblogic_access]
REPORT-weblogic_access = access-extractions

In transforms.conf I have

[testlookup]
filename = testing.csv

The inputlookup command shows that the lookup is being properly loaded, but a search of the form

sourcetype="weblogic_access" user != "-" | lookup testlookup user OUTPUTNEW username | table user, username

shows all of the users but the username column is blank. I have checked that the userids being looked up do exist in the lookup file!

What further troubleshooting can I do - ideally I'd like to get the lookup happening at search time with a LOOKUP-users stanza in props.conf but I suspect that fixing whatever is wrong here will correct the "Error in 'lookup' command: Could not find all of the specified lookup fields in the lookup table." message.

I get this problem whether I do it through the Manager or through the config files.

[shafqat571]

First make sure that you have assign proper user/app permission in bellow three areas
1 - Lookup table files
2 - Lookup definitions
3 - Automatic Lookups

If above three are OK. then
It is surely some wrong setting or typo in automatic lookups definition.

make sure that you have define proper setting under Lookups » Automatic lookups »
Lookup table definition [table definition]
correct sourcetype or host
proper input field maping
proper output field mapping

If all of the above are correct , lookup should be successful and you show see the result in fields side bar.

Or post sample logs and csv file

[willthames2]

With a little more digging, it seems that the lookup does work with the lookup command (one of the users that appears most in the logs isn't actually in the lookup table and so changes that improved the lookup appeared to have no effect until I dedupped the users).

However, it doesn't work with the automated lookup switched on:

Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'weblogic_access' and lookup table 'testlookup'

props.conf looks like:

[weblogic_access]
REPORT-weblogic_access = access-extractions
LOOKUP-weblogic_access = calling_app user AS s_account OUTPUTNEW calling_app

Actual CSV looks like

S_ACCOUNT,CALLING_APP
1234,userX
2345,userY

And we want to add a new field called calling_app based on the user field from weblogic_access, mapped to the s_account column in the lookup table (i.e. if user is 1234, calling_app should be userX)

[willthames2]

Ah, it should be

LOOKUP-weblogic_access = calling_app s_account AS user OUTPUTNEW calling_app

[willthames2]

the table lives in apps/$appname/lookups, the definition is in apps/$appname/transforms.conf, and the problem occurs when searching using the $appname app. No automated lookup as yet, but that would be in apps/$appname/props.conf

[Paolo_Prigione · Accepted solution]

You should probably check for the lookup access restrictions from the manager. From Manager --> Lookups:

• Lookup table files
• Lookup definitions

As a more generic suggestion, name the csv file after the transforms.conf stanza. I had handling problems when doing automated lookup updates with ... | outputlookup stanzaname.

[willthames2]

Read access is provided for everyone, and the permissions are now set to Global, but still without success. I've renamed the lookup csv file to be the same as the transforms stanza, and inputlookup still works, but not piping the search results to lookup

[hexx]

Is your lookup defined in the same app context that you are using for searching? If not, you will need to export all 3 lookup objects (the table, the lookuop definition and the automatic application) accordingly.