How to get the start and end time based on key words in logs?

[merc14]

Hi folks looking for some expert opinion.

my logs contains many diff files. I want to capture the start and end time for each file 

the logs looks like this

timestamp 202301_filex_a_b.z started execution

timestamp 202301_filex_a_b.z finished execution

timestamp 202301_filey_e_f.z started execution

timestamp 202301_filey_e_f.z finished execution

The output would look something like

filex | start timestamp | end timestamp | duration

filey | start timestamp | end timestamp | duration

I was able to do write diff search for start and end and then join them on the filename, but wondering if there is a better way to do it

[bowesmana · Accepted solution]

Simple method is

| stats min(_time) as start max(_time) as end by file
| eval duration=end-start

That assumes the following

• you have a field "file" containing the file name
• _time is the log timestamp of the event
• there are only 2 log messages per file and start always comes before end

It simply calculates the minimum and maximum value for the time and then calculates duration

[merc14]

need one more clarification, here file is a substring (filex, filey), can you please let me know how I can get the value for file  and combine it with | stats 

[bowesmana]

Use rex to extract the file name portion from the string that you want.

For example, if you have the _raw string which contains your data as in your example, you can do this regular expression to extract the filex/filey parts

| rex " \d{6}_(?<file>[A-Za-z0-9]+)"

that looks for a space + 6 digits then an _ before it then extracts a new field called "file" containing just the characters in the square brackets.

If you already have a field containing that entire string, then use

| rex field=your_field "\d{6}_(?<file>[A-Za-z0-9]+)"

or change the regex as needed.