How to find time between similar events?

[dl-it-serveradm]

Hello,

We are looking to create a search that will return when two similar events occur within 1 second of each other.

Sample log search results:

2022-04-19 18:42:39,210 INFO [stdout] (default task-43) [core.service.RestService] ==============POST Send Family=============
2022-04-19 18:39:31,142 INFO [stdout] (default task-43) [core.service.RestService] ==============POST Send Family=============
2022-04-19 18:35:38,403 INFO [stdout] (default task-41) [core.service.RestService] ==============POST Send Family=============
2022-04-19 18:35:38,371 INFO [stdout] (default task-42) [core.service.RestService] ==============POST Send Family=============
2022-04-19 18:34:01,696 INFO [stdout] (default task-40) [core.service.RestService] ==============POST Send Family=============
2022-04-19 18:30:36,450 INFO [stdout] (default task-39) [core.service.RestService] ==============POST Send Family=============
2022-04-19 16:57:39,144 INFO [stdout] (default task-36) [core.service.RestService] ==============POST Send Family=============
2022-04-19 14:01:42,904 INFO [stdout] (default task-153) [core.service.RestService] ==============POST Send Family=============
2022-04-19 13:46:00,629 INFO [stdout] (default task-153) [core.service.RestService] ==============POST Send Family=============
2022-04-19 13:42:39,944 INFO [stdout] (default task-153) [core.service.RestService] ==============POST Send Family=============
2022-04-19 13:32:59,488 INFO [stdout] (default task-145) [core.service.RestService] ==============POST Send Family=============

We would like a query to be able to return results when events occur, like the following times, since they are so close together:

2022-04-19 18:35:38,403 INFO [stdout] (default task-41) [core.service.RestService] ==============POST Send Family=============
2022-04-19 18:35:38,371 INFO [stdout] (default task-42) [core.service.RestService] ==============POST Send Family=============

Is there a way we can generate a query that would find something like that?  

Thanks!

[PickleRick · Accepted solution]

You need to use

streamstats window=1 current=f  values(_time) as prevtime

to find timestamp of previous occurrence of your event (you can add a "by" clause if you can distinguish different kinds of "similar" events). Then you can only filter the ones

 where _time - prevtime <1

The downside to this method is that you'll only get the second event of the two this way (or, if there are many subsequent quickly happening events - all of them except the first one).

You could also fiddle with the transaction command but it has its drawbacks and limitation.

[dl-it-serveradm]

Hi, this is very close.  I'm having trouble calculating the "_time - prevtime".

The prevtime is calculating the following values:

<query> | streamstats window=1 current=f values(_time) as prevtime | table _raw prevtime

2022-04-19 18:39:31,142 INFO [stdout] (default task-43) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650408159.210
2022-04-19 18:35:38,403 INFO [stdout] (default task-41) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650407971.142
2022-04-19 18:35:38,371 INFO [stdout] (default task-42) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650407738.403
2022-04-19 18:34:01,696 INFO [stdout] (default task-40) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650407738.371
2022-04-19 18:30:36,450 INFO [stdout] (default task-39) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650407641.696
2022-04-19 16:57:39,144 INFO [stdout] (default task-36) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650407436.450
2022-04-19 14:01:42,904 INFO [stdout] (default task-153) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650401859.144
2022-04-19 13:46:00,629 INFO [stdout] (default task-153) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650391302.904
2022-04-19 13:42:39,944 INFO [stdout] (default task-153) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650390360.629
2022-04-19 13:32:59,488 INFO [stdout] (default task-145) [core.service.OpsManagerRestService] ==============POST Send Family============= 1650390159.944

When running this query, it still returns all events:

<query> | streamstats window=1 current=f values(_time) as prevtime | where _time-prevtime <1

Thanks again. 

[dl-it-serveradm]

Ignore prior post.  Was able to use this syntax:

<query> | streamstats window=1 current=f values(_time) as prevtime | eval "TimeDiff" = prevtime - _time | where TimeDiff<1

Thanks for your help!