- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi folks looking for some expert opinion.
my logs contains many diff files. I want to capture the start and end time for each file
the logs looks like this
timestamp 202301_filex_a_b.z started execution
timestamp 202301_filex_a_b.z finished execution
timestamp 202301_filey_e_f.z started execution
timestamp 202301_filey_e_f.z finished execution
The output would look something like
filex | start timestamp | end timestamp | duration
filey | start timestamp | end timestamp | duration
I was able to do write diff search for start and end and then join them on the filename, but wondering if there is a better way to do it
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Simple method is
| stats min(_time) as start max(_time) as end by file
| eval duration=end-start
That assumes the following
- you have a field "file" containing the file name
- _time is the log timestamp of the event
- there are only 2 log messages per file and start always comes before end
It simply calculates the minimum and maximum value for the time and then calculates duration
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Simple method is
| stats min(_time) as start max(_time) as end by file
| eval duration=end-start
That assumes the following
- you have a field "file" containing the file name
- _time is the log timestamp of the event
- there are only 2 log messages per file and start always comes before end
It simply calculates the minimum and maximum value for the time and then calculates duration
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
need one more clarification, here file is a substring (filex, filey), can you please let me know how I can get the value for file and combine it with | stats
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Use rex to extract the file name portion from the string that you want.
For example, if you have the _raw string which contains your data as in your example, you can do this regular expression to extract the filex/filey parts
| rex " \d{6}_(?<file>[A-Za-z0-9]+)"
that looks for a space + 6 digits then an _ before it then extracts a new field called "file" containing just the characters in the square brackets.
If you already have a field containing that entire string, then use
| rex field=your_field "\d{6}_(?<file>[A-Za-z0-9]+)"
or change the regex as needed.