Splunk Search
Highlighted

Combining two non-correlated fields in a table

Explorer

Hi,

We have some log files created by the job. There are some lines in the logs as:

(TID#0x000013d8) 2014-Jun-24 05:11:56.928 INFO: [Source: ETC_FRW_alertLevel : Source] #52321 - Activation arguments - '{ modelId=ETC_WSH, processDate=23/6/2014 00:00:00.000000, aggUnit=default, runConfiguration=13 }'

and after some 10-15 lines there is a line as:

(TID#0x00001054) 2014-Jun-24 05:12:24.707 INFO: [Source: ETC_FRW_alertLevel : Source] #53007 - Source step 0 (Passed: 0, Filtered:0, Erroneous: 0, Containers deadlocks: 0).

I need the values corrsponding to runConfiguration i.e.13 in this case and Passed which is 0 in this case on the same line of the table.

I have made the search query but it is giving me values in two different lines as two lines are considered as two different events in Splunk. They do not even have common fileds in which we can join the 2 events.
The search query is:

| multisearch
 [search "passed" index=actim* | rex field=_raw "Passed: (?<etc_frw_passed>[0-9]+)" ]
 [search "runconfiguration" index=actim* | rex field=_raw "runConfiguration=(?<run_config>[0-9]+)"]
| table run_config etc_frw_passed _time source

Please suggest something.

Thanks.

Tags (3)
Highlighted

Re: Combining two non-correlated fields in a table

SplunkTrust
SplunkTrust

Is there any other way of correlating these two lines together? For example, does such a pair happen closely together in terms of time with a reasonably large gap to the next pair?

0 Karma
Highlighted

Re: Combining two non-correlated fields in a table

Communicator

I'm assuming these are from the same host and source log file? Without anything to exactly tie the events together, you can only guess that events that appear after each other are related. If that is ok, you could do this:

index=actim* ("passed" OR "runconfiguration") | rex field=_raw "Passed: (?<etc_frw_passed>[0-9]+)" | rex field=_raw "runConfiguration=(?<run_config>[0-9]+)"] | transaction host source maxevents=2 maxspan=60s startswith="run_config=*" endswith="etc_frw_passed=*" | table _time host source run_config etc_frw_passwd

View solution in original post

Highlighted

Re: Combining two non-correlated fields in a table

Explorer

Thanks Colin. This worked very well.

Yes these events are from same host and source.
it wouldbe great if you could explain me how you get that worked!!

Thanks

Highlighted

Re: Combining two non-correlated fields in a table

Communicator

Hi, If you could mark my answer as accepted that would be great!

The search works by building a base search that has all the events ( index=actim* ("passed" OR "runconfiguration") ). It then uses the transaction command to build a correlating transaction that (1) have exactly 2 events (2) Occur no more than 60 seconds apart (3) The first event needs to have your runconfig extracted field (4) The last event needs to have your etcfrw_passed extracted field.

Highlighted

Re: Combining two non-correlated fields in a table

Explorer

Hi,

I have another requirement of displaying the classification of the run configuration like for run configuration values ="11,21,2,5,9,41,20,25" the name to be displayed is 'TAS' similarly for run configuration values=="13,19,16,18" name should be 'EDW'.

The sample search query i have made is :

index=actim* ("passed" OR "runconfiguration") | rex field=raw "Passed: (?<etcfrwpassed>[0-9]+)" | rex field=raw "runConfiguration=(?[0-9]+)"] | transaction host source maxevents=2 maxspan=60s startswith="runconfig=*" endswith="etcfrwpassed=*" | eval runConfig= case(runconfig=18,"EDW",runconfig=11,"TAS") |table _time host source runconfig etcfrwpasswd

but no expected result.

Please suggest.

Thanks

Highlighted

Re: Combining two non-correlated fields in a table

SplunkTrust
SplunkTrust

Try this

   index=actim* ("passed" OR "runconfiguration") | rex field=_raw "Passed: (?<etc_frw_passed>[0-9]+)" | rex field=_raw "runConfiguration=(?<run_config>[0-9]+)"] | transaction host source maxevents=2 maxspan=60s startswith="run_config=*" endswith="etc_frw_passed=*" | eval run_config_name=case(LIKE(",11,21,2,5,9,41,20,25,","%,".run_config.",%"),"TAS",LIKE(",13,19,16,18,","%,".run_config.",%"),"EDW",1=1,"N/A")| table _time host source run_config run_config_name etc_frw_passwd

Note that there are commas at the start and end of the possible values (the string in case command) for run_config.

Highlighted

Re: Combining two non-correlated fields in a table

Explorer

Thanks a lot.This worked.

I didnot understood the meaning of "%,".run_config.",%" in this search.

0 Karma
Highlighted

Re: Combining two non-correlated fields in a table

Explorer

Hi,

Can you please explain me the LIKE(",13,19,16,18,","%,".run_config.",%") statement.

Thanks

0 Karma
Highlighted

Re: Combining two non-correlated fields in a table

SplunkTrust
SplunkTrust

% is wildcard character for WHERE..LIKE command. There first argument ",13,19,16,18,", each number/runconfig starts and ends with comma, so when comparing this set of numbers with your runconfig value, I added "%," as prefix and ",%" as suffix to it so that it will return true if run_config value matches any number in the set.

0 Karma