I am trying to use the transaction command to group events within 5 minutes of each other, and have set up fields to show start time, end time, and duration. With the way I've written my search, my end time is the start time of the last event, not the actual end time, since the last event has it's own duration.
How can I get the transaction command to set the end time as the end time of the last event, with the duration of the event added to the total transaction as well?
I found this problem when there was a single event for a particular user, and start time and end time were the same, since there was not another event to go along with it. This also resulted in the duration being 0, even though that was not correct.
Here's my current search:
index search ... | transaction maxpause=5m src_user | eval "endtime"=_time+duration | convert ctime("endtime") | eventstats sum(bytes_in) as "tbytesin" by _cd | eventstats sum(bytes_out) as "tbytesout" by _cd | addtotals fieldname=tbytes tbytes* | eval cduration=tostring(duration, "duration") | eval tbytesin = tostring(tbytesin, "commas") | eval tbytesout = tostring(tbytesout, "commas") | eval tbytes = tostring(tbytes, "commas") | eval tbytesin = "_ " + tbytesin | eval tbytesout = "_ " + tbytesout | eval tbytes = "_ " + tbytes | table src_user,_time,endtime,cduration,src_ip,tbytesin,tbytesout,tbytes | sort _time desc | convert timeformat="%m/%d/%Y %H:%M:%S" ctime(_time) | rename _time AS "Start Time",endtime AS "End Time",cduration as Duration,src_user AS "User",src_ip AS "IP Address",tbytesin AS "Total Bytes In",tbytesout AS "Total Bytes Out",tbytes AS "Total Bytes"
Can anyone help me with the last piece to this puzzle?
Not all events have a "duration", per se. You need to look at the form of the end event, and see if it has any information about duration, and if so, what field it might be in.
You should combine your two eventstats commands. They are grouping on the same field, so there is no reason to pass the file twice for that.
| eventstats sum(bytes_in) as "tbytesin", sum(bytes_out) as "tbytesout" by _cd
You are formatting tbytesin in two different lines which can be combined easily...
| eval tbytesin = "_ ".tostring(tbytesin, "commas")
Just as a general practice, its best to get into the habit of formatting the date in an order that can be compared directly:
"%Y-%m-%d" You will save yourself loads of grief that way, and no one in the world will ever wonder whether
Feb 4 or
Apr 2 if it is displayed as
Thanks for the tips!
From my understanding, the transaction command is grouping events specified on the arguments given. In my case, I'm wanting to group events within 5 minutes of each other, listing by src_user. Each of these events that get grouped in will have a duration from the transaction command, and I'm getting the end time from adding the duration to the start time.
... | transaction maxpause=5m src_user | eval "endtime"=_time+duration
So with that being said, each of the events would have a duration. Am I missing something here?
No each event does NOT have a duration by default. The transaction command groups a set of events into a new event, where the duration reflects the number of seconds between the timestamp of the first (sub)event in the transaction and the last (sub)event in the transaction. The timestamp of the transaction (_time) is set to the timestamp of the earliest/first (sub)event in the transaction.
And there is a way to get the individual fields from the (sub)events - if there is anything worth getting. And that was our question: do the sub(events) individually have a field that shows the elapsed time (duration)?
I was able to get around this issue by using eventstats after the transaction to show the first start time and last end time by _cd. Then, I converted the "true start/end" times to epoch format and did an eval for a "true duration" by doing a "trueendtime" - "truestarttime."
Thanks for all the help.
When you use the transaction command, "duration" will give you the duration in seconds. I've used tostring to convert it into a more readable format.
I don't know how to distinguish the last event using transaction, so I'm not sure how to get the last event's duration to be added to the total transaction duration.