How to extract two different fields on same timestamp based on the type of log statement?

[mlprasad79]

Hi there,
I have the following two different sample logger statements, the first statement written at the beginning of the process flow and the second logger is written at the end of the process flow.
1. [Info] 2017/09/09 12:00:00, 456 "Starting of the process"
2. [Info] 2017/09/09 12:00:00, 789 "End of the process".

Now, I want to extract 2 different fields on the time stamp, if the logger statement is "Starting of the process" I need to create beginTime field, and if the logger statement is "end of the process", I need to create endTime field, Please help.

[horsefez]

Hi mlprasad,

look at this solution.

| rex field=_raw "\]\s+?((?<begin_time>.+?)\s\"Starting|(?<end_time>.+?)\s\"End)"

After that you are able to reformat the newly created fields further.

[sbbadri]

@mlprasad79

try this,

your search | rex field=_raw "[\S+\s+]\s(?P<end_time>\d+-\d+-\d+\s\d+:\d+:\d+\,\d+)\s+-\s+.+(?P<MainFlowOUT>MainflowOUT)" | rex field=_raw "[\S+\s+]\s(?P<begin_time>\d+-\d+-\d+\s\d+:\d+:\d+\,\d+)\s+-\s+.+(?P<MainFlowIN>MainflowIN)"

[mlprasad79]

Hi Badri,
Thanks for your reply,

The query is kinda working fine, now I am trying to figure out the response time which is end_time - begin_time.
For that I am using the below query,
--above query--|eval response_time=strptime(end_time,"%Y-%m-%d %H:%M:%S.%3N") - strptime(begin_time,"%Y-%m-%d %H:%M:%S.%3N") |table begin_time, end_time , response_time.

but the response_time column is coming empty, not sure what went wrong.

[sbbadri]

Try this
strptime(end_time,”%Y/%m/%d %H:%M:%S, %3N”) and do the same thing for begin_time

[alemarzu]

Hello there @mlprasad79

This might work.

... | rex "\]\s(?<beginTime>[\d\/\s:]+)(?=,\s\d+\s\"Starting)" | rex "\]\s(?<endTime>[\d\/\s:]+)(?=,\s\d+\s\"End)"

Hope it helps.

[mlprasad79]

Hi @Alemarzu,

Thanks for your reply,
The rex is giving result till this portion | rex "]\s(?[\d\/\s:]+)(?=,\s\d+\s , but if I append \"Starting, it is producing empty results, what went wong??

[valiquet]

It would be more efficient to do both extractions in a single regex.

[alemarzu]

My bad, square brackets at the beginning were not scaped. Its fixed now.

[mlprasad79]

Hi Alemarzu,

Here is my actual sample first and last logger statements,
[INFO ] 2017-11-02 10:58:16,071 - com.aetna.eie.vtwoprovider.helper.util VTwoProviderRule 87675606-ddcc-4841-a925-96aac6a1a395-L7 MainflowOUT Exit the Ruleflow

[INFO ] 2017-11-02 10:58:16,071 - com.aetna.eie.vtwoprovider.helper.util VTwoProviderRule 87675606-ddcc-4841-a925-96aac6a1a395-L7 MainflowIN Into the Ruleflow

where the highlighted is the string decided whther the logger is first statement or last.

Here is my search query,
((com.aetna.eie.vtwoprovider.helper.util VTwoProviderRule * "MainflowIN Into the Ruleflow") OR (com.aetna.eie.vtwoprovider.helper.util VTwoProviderRule * "MainflowOUT Exit the Ruleflow")) |rex "]\s(?P\d+-\d+-\d+\s+\d+:\d+:\d+,\d+)(?=\s+-\s+\w+.\w+.\w+.\w+.\w+.\w+\s+\w+\s+ )"
if I try to hard code "MainflowIN" or "MainflowOUT" at the end of the query the results are blank, if I don't add this string results are coming but my ultimate goal is not achevied, please help.

[alemarzu]

Oh I see what happened. The log sample that you provided at the beginnig is not the same like the one above.