We are hitting a wall here... we would like to show events where a user does more than x actions within a small time window (1-3 seconds).
The log looks like this:
24.08.17;10:13:13|WebContainer : 36|DEBUG|User1|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: ENVP, Timestamp: 2017-08-24-10.13.13.0018 24.08.17;10:13:14|WebContainer : 36|DEBUG|User1|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: ENVP, Timestamp: 2017-08-24-10.13.13.0019 24.08.17;10:12:59|WebContainer : 37|DEBUG|User2|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: SAVE, Timestamp: 2017-08-24-10.12.59.0595 24.08.17;10:12:55|WebContainer : 5|DEBUG|User2|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: ENVP, Timestamp: 2017-08-24-10.12.55.0820 24.08.17;10:12:41|WebContainer : 0|DEBUG|A010999|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: ENVP, Timestamp: 2017-08-24-10.12.40.0992
we want to filter out the events where the delta between those events are more that x seconds... for the distinct user. Like above: only user1 has events within 1 seconds.
What we need now is something like:
User: EventCount relativetimdeltabetweenevents (might not be needed because its a filter condition)
User1 2 1 secs
how can we do this?
streamstats has a
time_window attribute. With that, you have a sliding time window and can do a count by user and action (if that is what you need, otherwise you can just do a count by user). Docs here.
There are a handful of ways. In addition to jeffland's answer above, you could also...
... | transaction maxspan=1m User | search duration>X
That will group events with the same User (and not separated by more than 1 minute - adjust that) into a "set" of events. You then have a duration, eventcount, and other information.
More performance (especially in a distributed environment) can be had by converting that to a stats command.
... | stats max(_time) as Latest min(_time) as Earliest BY User | eval duration=Latest-Earliest | search duration>X
You'll like need a bit more
list(fieldname) or 'values(fieldname)` in the stats command to get all your information out, but know it's possible. 🙂