Splunk Search

Extract field from a complex multi-lines event from log file

langlv
Engager

Hi pros,
I am new with Splunk and trying to analyze a complex log file from a financial application. I want to figure out the fields from a multilines event, here is my log example:

16.02.10 09:20:53 [ FromIso:123456789]************** INBOUND MESSAGE ID[AAABqgAwV0ujhQAA] ***************
 in[ 48: ]<800>
 in[ 48: ]<8220000100000000>
 in[ 48: ]<0400000000000000>
msgno[  0]<800>
Bitmap: [82200001000000000400000000000000]
 in[  7: ]<530>
 in[  7: ]<92833>
 in[ 11: ]<694437>
 in[ 32: ]<6>
 in[ 32: ]<123123>
 in[ 70: ]<2>
D-ISO-0306: m0800/a0000000000/t694437/p000000/r00
16.02.10 09:20:53 [   ToIso:123456789]************** OUTBOUND MESSAGE ID[AAABqgAwV0ujhQAA] ***************
msgno[  0]<810>
Bitmap: [82200001020000000400000000000000]
out[ 48: ]<0810>
out[ 48: ]<8220000102000000>
out[ 48: ]<0400000000000000>
out[  7: ]<0530>
out[  7: ]<092833>
out[ 11: ]<694437>
out[ 32: ]<06>
out[ 32: ]<123123>
out[ 39: ]<00>
out[ 70: ]<002>

I want to extract the fields msgno=800/810 and field39="00" from the log above.

I tried to use Field Extractor to extract those fields but no luck.

Thanks,
Lang

Tags (1)
0 Karma
1 Solution

woodcock
Esteemed Legend

At search time, like this:

... | rex max_match=99 "msgno\[\s*\d+\]<(?<msgno>\d+)>"
    | rex max_match=99 "out\[\s*39:\s*\]<(?<field39>\d+)>"

View solution in original post

0 Karma

woodcock
Esteemed Legend

At search time, like this:

... | rex max_match=99 "msgno\[\s*\d+\]<(?<msgno>\d+)>"
    | rex max_match=99 "out\[\s*39:\s*\]<(?<field39>\d+)>"
0 Karma

langlv
Engager

It works like a charm.
Thanks Woodcock,

0 Karma

kristian_kolb
Ultra Champion

Probably the best way is to use EXTRACTs in props.conf with the following regexes;

msgno[^<]+(?<msgno>\d+)
\s39:\s\]\<(?<field39>\d+)

You should probably check the documentation regarding EXTRACT

http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Createandmaintainsearch-timefieldextract...
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Also, it may be worth checking out the rex command, which lets you perform regex-based field extractions as part of the search query. At least it is simpler for trying out new extraction patterns before committing them to a config file.

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Rex

/k

Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...