Splunk Search

How do I search by multiple lines in a log?

Explorer

Right now, my search looks like this:

index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?201,.*)$"  | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ContainerID=mvindex(Msg,13) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest | table _time ActualDest ContainerID

and the log looks like this:

2016 05 09 12:32:29.000 | SDA written: 201,64,5,1,0,8,0,0,0,0,0,16790
2016 05 09 12:32:29.000 | 5,8,04S05577

I can get the destination that I need, but the container ID, 04S05577, doesn't get listed in the table. Help?

Thanks!

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Give this a try
Updated

index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$"  | rex "\|\s+([^,]+),([^,]+),(?<ContainerID>.+)$" | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5)  | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time 

Update #2

Feeling positive about this one

index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$"  | rex max_match=2 "\|\s+(?<temp>.+)$" 
| eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5)  | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest 
| eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time 

Update #3

 index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$" | rex max_match=2 "([\r\n]+)(?<temp>.+)"  | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5)  | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest 
    | eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time 

View solution in original post

SplunkTrust
SplunkTrust

Give this a try
Updated

index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$"  | rex "\|\s+([^,]+),([^,]+),(?<ContainerID>.+)$" | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5)  | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time 

Update #2

Feeling positive about this one

index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$"  | rex max_match=2 "\|\s+(?<temp>.+)$" 
| eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5)  | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest 
| eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time 

Update #3

 index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$" | rex max_match=2 "([\r\n]+)(?<temp>.+)"  | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5)  | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest 
    | eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time 

View solution in original post

Explorer

What I'm thinking now is that we try to isolate by the second line and ignore the first, eliminating the firstword=SDA.
Something like:

index=4_ip_cnv source="*ATL*Pack*" | rex "\s(?201,.*)$"  | rex max_match=2 "\|\s+(?.+)$" 
 | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5)  | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest 
 | eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDest) as ActualDest values(ContainerID) as ContainerID by _time

This gets closer to working. Now all I would have to do is eliminate any value in ContainerID that does not contain "04S"

0 Karma

SplunkTrust
SplunkTrust

You can add following ad the end of currently working search

...| eval ContainerID=mvfilter(match(ContainerID,"04S"))

Explorer

THANKS!!! If you're interested, it ended up being:

index=4_ip_cnv source="*ATL*Pack*" | rex "\s(?201,.*)$"  | rex max_match=2 "\|\s+(?.+)$" 
 | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5)  | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest 
 | eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDest) as ActualDest values(ContainerID) as ContainerID by _time| where like(ContainerID,"04%")| eval ContainerID=mvfilter(match(ContainerID,"04"))

Explorer

Not quite. It shows

2016-05-10 06:40:14 West 4 1789
2016-05-10 06:40:15 West 4 1790
2016-05-10 06:56:11 West 1 1791
2016-05-10 06:56:12 West 1 1792
2016-05-10 06:56:33 West 1 1793

Which is the correct destination, but does not include the container ID. Rather, it includes the 11th word of the first line.

0 Karma

Explorer

Unfortunately, not. Still does not display container ID. I think the misstep lies somewhere in rex "|\d+,\d+,(?\S+)$"

0 Karma

SplunkTrust
SplunkTrust

Looks like I missed taking the space after the pipe symbol. Just fixed the rex. Give that a shot.

0 Karma

Explorer

Nope 😕 same result.

0 Karma

SplunkTrust
SplunkTrust

😞
Try the updated query now (made changes to regex)

0 Karma

Explorer

That will just give me the entire first line after the 201 limit, I think.

0 Karma

SplunkTrust
SplunkTrust

Ok.. I probably wasn't clear in asking earlier. Does both the lines are part of single event?

e.g.
Event 1
016 05 09 12:32:29.000 | SDA written: 201,64,5,1,0,8,0,0,0,0,0,16790
2016 05 09 12:32:29.000 | 5,8,04S05577

Event2
016 05 09 12:32:29.000 | SDA written: 201,64,5,1,0,8,0,0,0,0,0,16790
2016 05 09 12:32:29.000 | 5,8,04S05577

0 Karma

Explorer

Yes, both lines are part of a single event, they just are written down into the log in separate lines.

0 Karma

SplunkTrust
SplunkTrust

The sample logs that you provided is one log entry OR two? You're using 14th element in the Msg field as ContainerID, but I don't see 14 elements which are separated by comma, So what is the location of ContainerID (is it last element in Msg?).

0 Karma

Explorer

It's in a separate line. I want to be able to include it. But you're right, I'm trying to include the 14th element, even though there isn't one. How can I include the second line to display that container ID?

0 Karma

Explorer

04S05577 is it, in the second separated line.

0 Karma

SplunkTrust
SplunkTrust

To be able to group these two events together, we need to find a rule/pattern. I don't see any common field between these two events, so can time be the key using which these can be joined together (along with other metadata fields)?

0 Karma

Explorer

Time can absolutely be used; these two messages will always show up at the exact same time.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!