Splunk Search

Need help with lookup query

Mrig342
Communicator

Hi All,

I am using below query to search for certain logs:

index=int_gcg_apac_solace_166076 host="mwgcb-csrla0*U*" source="/logs/confluent/connect-distributed/apac/TW/*" "Task is being killed and will not recover until manually restarted" | rex field=_raw "(?ms)id\=(?P<Connector>(\w+\.){1,9}\w+\-\d)\}" | lookup region_lookup.csv "source"

But while using the command | lookup region_lookup.csv "source", its not getting me any result based on the lookup table for the Region. 

I am trying to create a query using lookup table which will be as below:

sourceRegion
/logs/confluent/connect-distributed/apac/HK/*HongKong
/logs/confluent/connect-distributed/apac/SG/*Singapore
/logs/confluent/connect-distributed/apac/AU/*Australia
/logs/confluent/connect-distributed/apac/VN/*Vietnam
/logs/confluent/connect-distributed/apac/MY/*Malaysia
/logs/confluent/connect-distributed/apac/ID/*Indonesia
/logs/confluent/connect-distributed/apac/TH/*Thailand
/logs/confluent/connect-distributed/apac/TW/*Taiwan

 

Note: Each source have multiple folders inside it e.g. "logs/confluent/connect-distributed/apac/TW/*" will have file paths like "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log", "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log1", "logs/confluent/connect-distributed/apac/TW/kafkaconnect.log2" and so on..  And the searched indicator "Task is being killed and will not recover until manually restarted" may go into any of the folders.

Is there any way I can use, so that I can use the lookup table as desired..?
Your kind advise will be highly appreciated..

Thank You..!!

Labels (1)
0 Karma
1 Solution

ITWhisperer
Legend

You could try creating a shortened version of the source field which matches the format you have in your lookup file

index=int_gcg_apac_solace_166076 host="mwgcb-csrla0*U*" source="/logs/confluent/connect-distributed/apac/TW/*" "Task is being killed and will not recover until manually restarted" | rex field=_raw "(?ms)id\=(?P<Connector>(\w+\.){1,9}\w+\-\d)\}"
| rex field=source "^(?<src>(/[^/]+){5})"
| eval src=src."/*" 
| lookup region_lookup.csv src

View solution in original post

ITWhisperer
Legend

You could try creating a shortened version of the source field which matches the format you have in your lookup file

index=int_gcg_apac_solace_166076 host="mwgcb-csrla0*U*" source="/logs/confluent/connect-distributed/apac/TW/*" "Task is being killed and will not recover until manually restarted" | rex field=_raw "(?ms)id\=(?P<Connector>(\w+\.){1,9}\w+\-\d)\}"
| rex field=source "^(?<src>(/[^/]+){5})"
| eval src=src."/*" 
| lookup region_lookup.csv src

View solution in original post

Mrig342
Communicator

Thank you very much ITWhisperer..!!  You Rock..!!

The query worked perfect and I am able to get my desired output.

Thanks again.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!