Splunk Search

How to display respective entries from two different logs based on a common extracted field value?

amiivas
Engager

Hi All,
I have two different sources of log and want to display respective entries from each source based on a extracted field value from the first log. For e.g:
Log 1: Jan 6 15:33:13 xxxxx : trans(2735890423)[response][247.116.54.12] gtid(2735890423😞 |Test|service|247.116.54.12|2f4ad7ae-a4f9-324d-8d1a-8d98b414c496|2735890423||||/rest/services|documentId
Note - the field that need to be extracted from this log is value of gtid(2735890423), which is extracted as tranid. (highlighted with bold font)
Log 2: Jan 6 15:33:13 xxxxx : trans(2316097519)[response] gtid(2735890423): |Test|service|transaction type|response||2f4ad7ae-a4f9-324d-8d1a-8d98b414c496|2735890423:2316097519|2018-01-06T15:33:13-08:00|5|86|86|success|200 OK

Requirement is, get the value of the dptranid from log 1 and search the other source log for respective entries. This has to be done dynamically, meaning the entry from log 1 has to be search from a different search param but the query has to be in such a way that it returns entry from both logs.

For eg: as of know we are using: index="log1" /rest/services --> which results in entries from log 1. then we manually select the tran id from the log and then use another search query to get the result from log 2. I want to write a single query for the same purpose.

Thanks.

Tags (1)
0 Karma
1 Solution

mayurr98
Super Champion

hey amiivas,

Try this run anywhere search :

| makeresults 
| eval _raw="Jan 6 15:33:13 xxxxx : trans(2735890423)[response][247.116.54.12] gtid(2735890423): |Test|service|247.116.54.12|2f4ad7ae-a4f9-324d-8d1a-8d98b414c496|2735890423||||/rest/services|documentId" 
| append 
    [| makeresults 
    | eval _raw="Jan 6 15:33:13 xxxxx : trans(2316097519)[response] gtid(2735890423): |Test|service|transaction type|response||2f4ad7ae-a4f9-324d-8d1a-8d98b414c496|2735890423:2316097519|2018-01-06T15:33:13-08:00|5|86|86|success|200 OK"] 
| rex field=_raw "gtid\((?P<trans_id>[^])]+)" 
| stats values(_raw) as raw_event by trans_id

If you want to try with raw logs then assuming that you have two seperate indexes i.e. index1 and index2 for both sources try this: NOTE: if you have only one index then write only index=your_index (source=source1 */rest/services* OR source=source2)

  index=index1  OR index=index2  (source=source1 */rest/services* OR source=source2)
    | rex field=_raw "gtid\((?P<trans_id>[^])]+)" 
    | stats values(_raw) as raw_event by trans_id

Let me know if that helps you!

View solution in original post

mayurr98
Super Champion

hey amiivas,

Try this run anywhere search :

| makeresults 
| eval _raw="Jan 6 15:33:13 xxxxx : trans(2735890423)[response][247.116.54.12] gtid(2735890423): |Test|service|247.116.54.12|2f4ad7ae-a4f9-324d-8d1a-8d98b414c496|2735890423||||/rest/services|documentId" 
| append 
    [| makeresults 
    | eval _raw="Jan 6 15:33:13 xxxxx : trans(2316097519)[response] gtid(2735890423): |Test|service|transaction type|response||2f4ad7ae-a4f9-324d-8d1a-8d98b414c496|2735890423:2316097519|2018-01-06T15:33:13-08:00|5|86|86|success|200 OK"] 
| rex field=_raw "gtid\((?P<trans_id>[^])]+)" 
| stats values(_raw) as raw_event by trans_id

If you want to try with raw logs then assuming that you have two seperate indexes i.e. index1 and index2 for both sources try this: NOTE: if you have only one index then write only index=your_index (source=source1 */rest/services* OR source=source2)

  index=index1  OR index=index2  (source=source1 */rest/services* OR source=source2)
    | rex field=_raw "gtid\((?P<trans_id>[^])]+)" 
    | stats values(_raw) as raw_event by trans_id

Let me know if that helps you!

amiivas
Engager

Hi mayurr98,

index=index1 OR index=index2 (source=source1 */rest/services* OR source=source2) | rex field=_raw "gtid\((?P<trans_id>[^])]+)" | stats values(_raw) as raw_event by trans_id

I was trying with raw logs and the above query only resulted in entries from Log 1.

But your anywhere search is working as what expected.

Thanks,

0 Karma

mayurr98
Super Champion

Okay the problem must be in initial filter search

Can you just run

index=index1  OR index=index2  ((source=source1 */rest/services*) OR source=source2))

See if you are able to get logs from both sources??
If yes then run rex command and see if trans_id is getting extracted from both searches and like this you can troubleshoot!

Let me know if this helps you!

0 Karma

amiivas
Engager

Thanks it worked.. Awesome thanks. Did some here and there, some extra brackets, some more filtering and it worked.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...