Time difference by grouping identical events

peeeeeeeeeeter · ‎09-22-2019

Suppose I have the following events.

2019-09-20 01:40:09 INFO Listener processing event with message key A1:B1:C1
2019-09-20 01:40:06 INFO Listener processing event with message key A1:B1:C1

2019-09-20 01:40:00 INFO Listener processing event with message key A1:B1:C2

2019-09-20 01:39:57 INFO Listener processing event with message key A1:B1:C2

The event patterns are exactly identical, and the events differ only by timestamp, and they come in pairs. The timestamps are the start and end time of the event.

I would like to generate a table to summarize the events, which looks like the following

Field1| Field2 | Field3| StartTime | EndTime | Duration
A1 | B1 | C1 | 2019-09-20 01:40:06 | 2019-09-20 01:40:09 | 3
A1 | B1 | C2 | 2019-09-20 01:39:57 | 2019-09-20 01:40:00 | 3

The main code block looks like the following:

source="*.log"
| rex field=_raw "message key (?<A>.*?):(?<B>.*?):(?<C>.*)"

| table A B C _time

I have tried both the transaction and stats function, but in vain, maybe I did not use them correctly.

Is there anyone who can give me some advice on what to do, any help will be highly appreciated!

DavidHourani · ‎09-23-2019

Hi peter with many e,

Try something like this:

 source="*.log"
 | rex field=_raw "message key (?<A>.*?):(?<B>.*?):(?<C>.*)"
 |stats earliest(_time) as StartTime latest(StartTime) as EndTime by A,B,C
 | eval duration= tostring(EndTime-StartTime,"duration")
 | table A B C StartTime EndTime duration

Let me know how that works out for you.

Cheers,
David