Splunk Search

Extract field from a complex multi-lines event from log file

langlv
Engager

Hi pros,
I am new with Splunk and trying to analyze a complex log file from a financial application. I want to figure out the fields from a multilines event, here is my log example:

16.02.10 09:20:53 [ FromIso:123456789]************** INBOUND MESSAGE ID[AAABqgAwV0ujhQAA] ***************
 in[ 48: ]<800>
 in[ 48: ]<8220000100000000>
 in[ 48: ]<0400000000000000>
msgno[  0]<800>
Bitmap: [82200001000000000400000000000000]
 in[  7: ]<530>
 in[  7: ]<92833>
 in[ 11: ]<694437>
 in[ 32: ]<6>
 in[ 32: ]<123123>
 in[ 70: ]<2>
D-ISO-0306: m0800/a0000000000/t694437/p000000/r00
16.02.10 09:20:53 [   ToIso:123456789]************** OUTBOUND MESSAGE ID[AAABqgAwV0ujhQAA] ***************
msgno[  0]<810>
Bitmap: [82200001020000000400000000000000]
out[ 48: ]<0810>
out[ 48: ]<8220000102000000>
out[ 48: ]<0400000000000000>
out[  7: ]<0530>
out[  7: ]<092833>
out[ 11: ]<694437>
out[ 32: ]<06>
out[ 32: ]<123123>
out[ 39: ]<00>
out[ 70: ]<002>

I want to extract the fields msgno=800/810 and field39="00" from the log above.

I tried to use Field Extractor to extract those fields but no luck.

Thanks,
Lang

Tags (1)
0 Karma
1 Solution

woodcock
Esteemed Legend

At search time, like this:

... | rex max_match=99 "msgno\[\s*\d+\]<(?<msgno>\d+)>"
    | rex max_match=99 "out\[\s*39:\s*\]<(?<field39>\d+)>"

View solution in original post

0 Karma

woodcock
Esteemed Legend

At search time, like this:

... | rex max_match=99 "msgno\[\s*\d+\]<(?<msgno>\d+)>"
    | rex max_match=99 "out\[\s*39:\s*\]<(?<field39>\d+)>"
0 Karma

langlv
Engager

It works like a charm.
Thanks Woodcock,

0 Karma

kristian_kolb
Ultra Champion

Probably the best way is to use EXTRACTs in props.conf with the following regexes;

msgno[^<]+(?<msgno>\d+)
\s39:\s\]\<(?<field39>\d+)

You should probably check the documentation regarding EXTRACT

http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Createandmaintainsearch-timefieldextract...
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Also, it may be worth checking out the rex command, which lets you perform regex-based field extractions as part of the search query. At least it is simpler for trying out new extraction patterns before committing them to a config file.

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Rex

/k

Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...