How to do regex Extraction on multiple lines?

[chidiuchegbu]

I have been avoiding RegEx for quite sometime in Splunk but I now I really need to deal with it and understand it.

I really need help with this three cases;

First Case:

I have events that usually start with things like 57A,53A and followed by other strings. I want to match 57A but the field value will be FMDKNTLA

:57A:3232324646
FMDKNTLA

Second Case:

Another extraction example will be extracting  field 31A but the value will be "NKN"

:32A:200117NKN200000000,00

Third Case:

extracting field 31A but the value will be "200000000,00"

:31A:200117NKN200000000,00

Any help will be appreciated

[ITWhisperer]

| makeresults
| eval _raw=":57A:3232324646 FMDKNTLA
:32A:200117NKN200000000,00
:32A:200117NKN200000000,00"
| multikv noheader=t
| table _raw
| eval _raw=replace(_raw," ","
")
``` the lines above set up data as posted ```
| rex ":57A:\d+\s*(?<case1>\w+)"
| rex ":32A:\d+(?<case2>[[:upper:]]+)(?<case3>[\d\,]+)"

[chidiuchegbu]

Hi ITWhisperer,

Thanks a lot. Case 2 and case 3 works but case 1 is not matching. Could you please review again? note that

 :57A:3232324646 FMDKNTLA are not on the same line in the logs but like the below

:57A:3232324646

FMDKNTLA

[ITWhisperer]

That's why I put the replace in when setting up the data to change the space to a new line.

I am assuming the data is still in one event?

[chidiuchegbu]

If you can put little explanation to each syntax that will really help

[chidiuchegbu]

Thank you. I will try it out again. Though I do not understand fully how it all come together. Its a good place to begin my learning.

Yes the data is on one event with other data --serveral lines , upto 20 lines but the below is the focus of this extraction, while the others will come separately since am not sure if all can in one rex..

................................................. 

:53A:/D/3232324646
FMDKNTLA
:57A:/D/3123424646
FTDMNSLA

so I will extract 53A data separately matching to dynamic data "FMDKNTLA" and same with 57A as you have rightly done

[ITWhisperer]

OK where did the /D/ come from? That wasn't in your original description of the events - this is why the extract doesn't match for the first case

[chidiuchegbu]

Oka it must have been wrongly copied from regex101 as I have been testing this there.

However, this content is entirely dynamic - meaning it can change to anything. So my focus

:57A:
FIDTNGLA

[ITWhisperer]

rex works by matching strings to find anchors so that strings can be extracted. This relies on there being a pattern. You have now given yet another version of this field. Please can you give more examples (a definitive list?) and explain which part you want extracting.

[chidiuchegbu]

I did not give another version. am just saying that the certain fields are not static while others are. at the end of the day what I want to extract is 

:57A:
FIDTNGLA

This is a sample event which I posted earlier...This is one event with many strings

:53A:/D/3232324646
FMDKNTLA
:57A:/D/3123424646
FTDMNSLA

So I need 53A as field and FMDKNTLA as the value 

and separately needs 57A as the field and FTDMNSLA.

Also there are other lines preceding the events above (same event) but I have successfully extracted them

hope this is clear 

[ITWhisperer]

| rex ":57A:\S*(?<case1a>\w+)"
| rex ":53A:\S*(?<case1b>\w+)"

[chidiuchegbu]

This is still not working and am not sure why. I have just modified sensitive info to paste the full event below.

${1:A29FDDTMLAXXX4444204444}{6:{107:200971602}{441:0}}{1:A29FDDTMLAXXX4444204444}{2:O1092343420017IKNITMLAAXXX45559999999001161692N}{3:{105:NGM}{141:0070}{111:c7837321-c49e-4ccc-b555-05492e60dfd5}{115:1230000}}{6:
:20:KKNXXX33333333
:23B:DREM
:23E:KDVA
:26T:001
:32A:200617NKN115003980,00
:51K:/2222222222
CAMA TECHNOLOGY COMPANY LIM
MITED
:53A:/D/0000024849
FMDKNTLA
:57A:/D/0000024946
FTDMNSLA
:59:/4440555666
MITRALO COMPANY
:71A:OWW
:72:/COMTYPTN/021
/BNN/CAMA TECHNOLOGY
-}{5:{LAC:00000000}{LAC:00000000}{KMK:0000000000}}{S:{KAC:}{DAC:}{WOP:S}}

[chidiuchegbu]

Note  that the number in between ":3232324646 "  for case 1 :57A:3232324646 FMDKNTLA  should be skipped 

only

 :57A - FMDKNTLA needed as field and value respectively but again 57A is on a different line from FMDKNTLA  represented in the raw log like the below

57A:3232324646
FMDKNTLA 

[ITWhisperer]

[chidiuchegbu]

I will try it out

[chidiuchegbu]

Thanks for the feedback. Yes I have been using regex101.com to learn.

Throwing more light to this. This is a multiple lines of strings of data.The only static field is the 57A,32A but the others are dynamic and changes with data

For use case 2 and 3

Only 32A is static field but the "NKN" and the digits before and after it changes with each data

32A:200117NKN200000000,00

for use case 1 below, only 57A is constant, "FMDKNTLA" is dynamic and can be "ASTGNMLA" with next data

:57A:3232324646
FMDKNTLA

So I want have it extracted as 

use case 1 - field and field value :  >>>57A - FMDKNTLA but can be "57A - ASTGNMLA" next time

use case 2 - field and field value: >>> 32A - NKN  but can be "32A - USD" next time

use case 3 - field and field value: >>> 32A - 200000000,00  but can be "32A - 3400000" next time

Dont know if this helps

[isoutamo]

Hi

here is some examples, maybe not the perfect ones as I haven't so enough samples what you have.

1) 

| makeresults 
| fields _time
| eval _raw = ":57A:3232324646
FMDKNTLA"
| rex "^:(?<field>[^:]+)"

2)  I'm exception that you are meaning 32A not 31A? In that case option 1 works also in this case.

3) Also in this case option 1 should work.

https://regex101.com is nice place to work & test those.

r. Ismo