Need help with query

[Mrig342]

Hi All,

I am trying to extract a field from the below log.

log1:

esbgc_as_uat2_dom|ESB/Monitoring/ESB_HealthCheck|esbgc_as_uat2_dom|Synchronized|Thu 29-Apr-2021 07:18:55 EST|1
esbgc_as_uat2_dom|ESB/Framework/Dispatcher/InterfaceDispatcher_SG_IDT2_CPG_A|esbgc_as_uat2_dom|Synchronized|Thu 29-Apr-2021 04:59:40 EST|2
esbgc_as_uat2_dom|ESB/Framework/Dispatcher/InterfaceDispatcher_SG_IDT2_CPG_B|esbgc_as_uat2_dom|Synchronized|Thu 29-Apr-2021 05:01:45 EST|2

I created the below query to extract the field "App_Name" which is "ESB_HealthCheck|esbgc_as_uat2_dom"from the log:

| rex field=_raw "^[^\|\n]*\|(?P<App_Name>[^\|]+)"

Here, I am getting the value from the line one only. How will I be able to extract the value from all the lines in the log.

Please help me creating the query to get the desired output.

Your kind support will be highly appreciated.

Thank you.

[ITWhisperer]

Are these lines all in one event? If so, you could use max_match=0 to give you a multi-value field of App_Names

[gcusello]

Hi @Mrig342,

let me understand: the rows you shared are from one event or from more events?

if they come from more events, your regex is correct,

if instead your rows come from one event, you have to modify your regex, something like this:

| rex "(?ms)^[^\|\n]*\|(?P<App_Name>[^\|]+)"

but I think that, if all the rows come from only one event, the problem isn't the field extraction, but the event separation because you have a timestamp in each row.

Your props.con reaches to divide the events?

Ciao.

Giuseppe

[Mrig342]

Hi @gcusello,

Thank you very much for your reply and suggestion. 

The lines are of one event. You can consider as below:

log1: 

esbgc_ss_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_ss_uat2_dom|Synchronized|Sat 01-May-2021 08:13:18 EST|1 esbgc_as_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_as_uat2_dom|Synchronized|Sat 01-May-2021 08:13:32 EST|1

log2:

esbgc_ss_uat2_dom|ESB/Services/ProductBaseV6/Prov_CardServices_6A|esbgc_ss_uat2_dom|Synchronized|Fri 30-Apr-2021 10:58:10 EST|1

etc. and some logs might have 3 or more lines too.

When trying with the query you gave ("(?ms)^[^\|\n]*\|(?P<App_Name>[^\|]+)"), I am getting output as:

esbgc_ss_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_ss_uat2_dom|Synchronized|Sat 01-May-2021 08:13:18 EST|1 esbgc_as_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_as_uat2_dom|Synchronized|Sat 01-May-2021 08:13:32 EST|1

• App_Name = ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A

Also I tried to give a max_match=0 to consider multivalues using query (| rex field=_raw "(?ms)^[^\|\n]*\|(?P<App_Name>[^\|]+)"), I am getting output as:

esbgc_ss_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_ss_uat2_dom|Synchronized|Sat 01-May-2021 08:13:18 EST|1 esbgc_as_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_as_uat2_dom|Synchronized|Sat 01-May-2021 08:13:32 EST|1

• App_Name = ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A App_Name = esbgc_ss_uat2_dom App_Name = Synchronized App_Name = Sat 01-May-2021 08:13:18 EST App_Name = 1 esbgc_as_uat2_dom App_Name = ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A App_Name = esbgc_as_uat2_dom App_Name = Synchronized App_Name = Sat 01-May-2021 08:13:32 EST App_Name = 1

Here, it is considering everything as App_Name.

Please help look into it and correct the query.

Thank you.

[ITWhisperer]

It is a little difficult to tell if there are line-breaks between the "lines" or whether it has simply word-wrapped. Assuming that there are line-breaks, (?m)^ takes care of them so you should only need to skip past the first pipe-delimited part on each line. 

| rex max_match=0 "(?m)^[^\|]*\|(?P<App_Name>[^\|]+)"

[Mrig342]

Hi @ITWhisperer,

Thank you for your suggestion.

However the query still gives output as before. 

esbgc_ss_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_ss_uat2_dom|Synchronized|Sat 01-May-2021 08:13:18 EST|1 esbgc_as_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_as_uat2_dom|Synchronized|Sat 01-May-2021 08:13:32 EST|1

• App_Name = ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A App_Name = esbgc_ss_uat2_dom App_Name = Synchronized App_Name = Sat 01-May-2021 08:13:18 EST App_Name = 1 esbgc_as_uat2_dom App_Name = ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A App_Name = esbgc_as_uat2_dom App_Name = Synchronized App_Name = Sat 01-May-2021 08:13:32 EST App_Name = 1

Seems line break is the issue. Considering to configure the line breaks and will try then.

If you can suggest anything else, it's welcomed.

Thank you very much for your support.

[ITWhisperer · Accepted solution]

Assuming there are line-breaks, try this:

| makeresults
| eval _raw="esbgc_ss_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_ss_uat2_dom|Synchronized|Sat 01-May-2021 08:13:18 EST|1
esbgc_as_uat2_dom|ESB/Services/ProductBaseV6/Prov_DepositsAndRatesServices_6A|esbgc_as_uat2_dom|Synchronized|Sat 01-May-2021 08:13:32 EST|1"
| rex max_match=0 "(?m)^[^\|]+\|(?P<App_Name>[^\|]+)"

[Mrig342]

Hi @ITWhisperer,

This query works fine. Thank you very much for your support.

[manjunathmeti]

hi @Mrig342,

Try this:

| rex "\/(?<App_Name>\w+\|\w+)"