Hi All,
I am using below query to search for certain logs:
index=int_gcg_apac_solace_166076 host="mwgcb-csrla0*U*" source="/logs/confluent/connect-distributed/apac/TW/*" "Task is being killed and will not recover until manually restarted" | rex field=_raw "(?ms)id\=(?P<Connector>(\w+\.){1,9}\w+\-\d)\}" | lookup region_lookup.csv "source"
But while using the command | lookup region_lookup.csv "source", its not getting me any result based on the lookup table for the Region.
I am trying to create a query using lookup table which will be as below:
source | Region |
/logs/confluent/connect-distributed/apac/HK/* | HongKong |
/logs/confluent/connect-distributed/apac/SG/* | Singapore |
/logs/confluent/connect-distributed/apac/AU/* | Australia |
/logs/confluent/connect-distributed/apac/VN/* | Vietnam |
/logs/confluent/connect-distributed/apac/MY/* | Malaysia |
/logs/confluent/connect-distributed/apac/ID/* | Indonesia |
/logs/confluent/connect-distributed/apac/TH/* | Thailand |
/logs/confluent/connect-distributed/apac/TW/* | Taiwan |
Note: Each source have multiple folders inside it e.g. "logs/confluent/connect-distributed/apac/TW/*" will have file paths like "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log", "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log1", "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log2" and so on.. And the searched indicator "Task is being killed and will not recover until manually restarted" may go into any of the folders.
Is there any way I can use, so that I can use the lookup table as desired..?
Your kind advise will be highly appreciated..
Thank You..!!
You could try creating a shortened version of the source field which matches the format you have in your lookup file
index=int_gcg_apac_solace_166076 host="mwgcb-csrla0*U*" source="/logs/confluent/connect-distributed/apac/TW/*" "Task is being killed and will not recover until manually restarted" | rex field=_raw "(?ms)id\=(?P<Connector>(\w+\.){1,9}\w+\-\d)\}"
| rex field=source "^(?<src>(/[^/]+){5})"
| eval src=src."/*"
| lookup region_lookup.csv src
You could try creating a shortened version of the source field which matches the format you have in your lookup file
index=int_gcg_apac_solace_166076 host="mwgcb-csrla0*U*" source="/logs/confluent/connect-distributed/apac/TW/*" "Task is being killed and will not recover until manually restarted" | rex field=_raw "(?ms)id\=(?P<Connector>(\w+\.){1,9}\w+\-\d)\}"
| rex field=source "^(?<src>(/[^/]+){5})"
| eval src=src."/*"
| lookup region_lookup.csv src
Thank you very much ITWhisperer..!! You Rock..!!
The query worked perfect and I am able to get my desired output.
Thanks again.