topic Re: extracting data using rex field=_raw in All Apps and Add-ons

How to extract data from log message data using rex field=_raw?

sandysaahil — Mon, 04 Apr 2022 14:49:51 GMT

Hi Gurus,

I am trying to extract data from log message using rex field=_raw. The regex I have is

"Event <(?<eventNo>.*)>, Super <(?<super>.*)>, Charge <(?<oic>.*)>, number <(?<pcn>.*)>, Card <(?<cn>.*)>, CO <(?<co>.*)>, Warn <(?<warn>.*)>" | table _time oic eventNo pcn cn super co warn

and I am able to extract records. but the issue is may or may bot be present is the log and I still need to extract the rest of the data. I tried

"Event <(?<eventNo>.*)>, Super <(?<super>.*)>, Charge <(?<oic>.*)>, number <(?<pcn>.*)>, Card <(?<cn>.*)>, (CO <(?<co>.*)>,)? Warn <(?<warn>.*)>" | table _time oic eventNo pcn cn super co warn

It gives me the records which does nto contain this item. I want to extract all the records irrestive of whether it is present or not present. Please let me know what am I doing wrong. Thanks a lot in advance.

Re: extracting data using rex field=_raw

bowesmana — Mon, 04 Apr 2022 01:23:28 GMT

Please provide examples of both types of data. I am assuming you are saying that CO may or may not be in the data...?

Re: extracting data using rex field=_raw

sandysaahil — Mon, 04 Apr 2022 01:37:50 GMT

You are right @bowesmana , the example data is

Event <E523478>, Super <0>, Charge <N46965>, number <0199327578>, Card <2898005466>, Warn <true>
Event <E523478>, Super <0>, Charge <N46965>, number <0199327578>, Card <2898005466>, Warn <false>
Event <E521244>, Super <2>, Charge <N46965>, number <0199327750>, Card <2898005471>, CO <true>, Warn <true>

In third case, I get the value of CO which is missing from other two log statements. I want to ignore CO in other two but would like to extract CO if available.

Re: extracting data using rex field=_raw

gcusello — Mon, 04 Apr 2022 06:54:38 GMT

Hi @sandysaahil,

in your logs you have a word thatr identifies each field, so you could create a regex for each field, in this way the other regexes aren't blocked when one field is missed, something like this:

Ciao.

Giuseppe

Re: extracting data using rex field=_raw

sandysaahil — Wed, 13 Apr 2022 04:27:24 GMT

@gcusello This is working perfectly fine. Thanks for the solution. Just want to check one more thing. one of the rex fields contains hypens. I tried using rex "Expiry \<(?[\d-]*)>" . It onle fetches first record but ignores rest of the same expiry fields from other logs. I get blank column for that. Do you think I am doing something wrong here. It works perfectly for all other words which does not contain special characters.

Re: extracting data using rex field=_raw

gcusello — Wed, 13 Apr 2022 06:22:40 GMT

Hi @sandysaahil,

I hint to create a new question, so more people can help you better and quicker!

Anyway, in the sample you shared there isn't any hypen, could you share some sample with and without hypens?

Ciao.

Giuseppe

P.S.: Karma Points are appreciated by all the Contributors 😉