Need help with lookup query

[Mrig342]

Hi All,

I am using below query to search for certain logs:

index=int_gcg_apac_solace_166076 host="mwgcb-csrla0*U*" source="/logs/confluent/connect-distributed/apac/TW/*" "Task is being killed and will not recover until manually restarted" | rex field=_raw "(?ms)id\=(?P<Connector>(\w+\.){1,9}\w+\-\d)\}" | lookup region_lookup.csv "source"

But while using the command | lookup region_lookup.csv "source", its not getting me any result based on the lookup table for the Region. 

I am trying to create a query using lookup table which will be as below:

source	Region
/logs/confluent/connect-distributed/apac/HK/*	HongKong
/logs/confluent/connect-distributed/apac/SG/*	Singapore
/logs/confluent/connect-distributed/apac/AU/*	Australia
/logs/confluent/connect-distributed/apac/VN/*	Vietnam
/logs/confluent/connect-distributed/apac/MY/*	Malaysia
/logs/confluent/connect-distributed/apac/ID/*	Indonesia
/logs/confluent/connect-distributed/apac/TH/*	Thailand
/logs/confluent/connect-distributed/apac/TW/*	Taiwan

Note: Each source have multiple folders inside it e.g. "logs/confluent/connect-distributed/apac/TW/*" will have file paths like "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log", "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log1", "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log2" and so on..  And the searched indicator "Task is being killed and will not recover until manually restarted" may go into any of the folders.

Is there any way I can use, so that I can use the lookup table as desired..?
Your kind advise will be highly appreciated..

Thank You..!!

[ITWhisperer · Accepted solution]

You could try creating a shortened version of the source field which matches the format you have in your lookup file

index=int_gcg_apac_solace_166076 host="mwgcb-csrla0*U*" source="/logs/confluent/connect-distributed/apac/TW/*" "Task is being killed and will not recover until manually restarted" | rex field=_raw "(?ms)id\=(?P<Connector>(\w+\.){1,9}\w+\-\d)\}"
| rex field=source "^(?<src>(/[^/]+){5})"
| eval src=src."/*" 
| lookup region_lookup.csv src

[Mrig342]

Thank you very much ITWhisperer..!!  You Rock..!!

The query worked perfect and I am able to get my desired output.

Thanks again.