Splunk Search

Extract field from a complex multi-lines event from log file

langlv
Engager

Hi pros,
I am new with Splunk and trying to analyze a complex log file from a financial application. I want to figure out the fields from a multilines event, here is my log example:

16.02.10 09:20:53 [ FromIso:123456789]************** INBOUND MESSAGE ID[AAABqgAwV0ujhQAA] ***************
 in[ 48: ]<800>
 in[ 48: ]<8220000100000000>
 in[ 48: ]<0400000000000000>
msgno[  0]<800>
Bitmap: [82200001000000000400000000000000]
 in[  7: ]<530>
 in[  7: ]<92833>
 in[ 11: ]<694437>
 in[ 32: ]<6>
 in[ 32: ]<123123>
 in[ 70: ]<2>
D-ISO-0306: m0800/a0000000000/t694437/p000000/r00
16.02.10 09:20:53 [   ToIso:123456789]************** OUTBOUND MESSAGE ID[AAABqgAwV0ujhQAA] ***************
msgno[  0]<810>
Bitmap: [82200001020000000400000000000000]
out[ 48: ]<0810>
out[ 48: ]<8220000102000000>
out[ 48: ]<0400000000000000>
out[  7: ]<0530>
out[  7: ]<092833>
out[ 11: ]<694437>
out[ 32: ]<06>
out[ 32: ]<123123>
out[ 39: ]<00>
out[ 70: ]<002>

I want to extract the fields msgno=800/810 and field39="00" from the log above.

I tried to use Field Extractor to extract those fields but no luck.

Thanks,
Lang

Tags (1)
0 Karma
1 Solution

woodcock
Esteemed Legend

At search time, like this:

... | rex max_match=99 "msgno\[\s*\d+\]<(?<msgno>\d+)>"
    | rex max_match=99 "out\[\s*39:\s*\]<(?<field39>\d+)>"

View solution in original post

0 Karma

woodcock
Esteemed Legend

At search time, like this:

... | rex max_match=99 "msgno\[\s*\d+\]<(?<msgno>\d+)>"
    | rex max_match=99 "out\[\s*39:\s*\]<(?<field39>\d+)>"
0 Karma

langlv
Engager

It works like a charm.
Thanks Woodcock,

0 Karma

kristian_kolb
Ultra Champion

Probably the best way is to use EXTRACTs in props.conf with the following regexes;

msgno[^<]+(?<msgno>\d+)
\s39:\s\]\<(?<field39>\d+)

You should probably check the documentation regarding EXTRACT

http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Createandmaintainsearch-timefieldextract...
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Also, it may be worth checking out the rex command, which lets you perform regex-based field extractions as part of the search query. At least it is simpler for trying out new extraction patterns before committing them to a config file.

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Rex

/k

Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...