Splunk Search

Calculate time difference between events for distinct users in transaction log

hgehrts_splunk
Splunk Employee
Splunk Employee

Hi

We are hitting a wall here... we would like to show events where a user does more than x actions within a small time window (1-3 seconds).

The log looks like this:

24.08.17;10:13:13|WebContainer : 36|DEBUG|User1|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: ENVP, Timestamp: 2017-08-24-10.13.13.0018
24.08.17;10:13:14|WebContainer : 36|DEBUG|User1|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: ENVP, Timestamp: 2017-08-24-10.13.13.0019
24.08.17;10:12:59|WebContainer : 37|DEBUG|User2|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: SAVE, Timestamp: 2017-08-24-10.12.59.0595
24.08.17;10:12:55|WebContainer : 5|DEBUG|User2|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: ENVP, Timestamp: 2017-08-24-10.12.55.0820
24.08.17;10:12:41|WebContainer : 0|DEBUG|A010999|d.s.s.c.BaseController|SomeController.java:2269 - MyHandler: 561170, MyStep: 1801513, Action: ENVP, Timestamp: 2017-08-24-10.12.40.0992

we want to filter out the events where the delta between those events are more that x seconds... for the distinct user. Like above: only user1 has events within 1 seconds.

What we need now is something like:
User: EventCount relativetimdeltabetweenevents (might not be needed because its a filter condition)
User1 2 1 secs

how can we do this?

0 Karma
1 Solution

jeffland
SplunkTrust
SplunkTrust

streamstats has a time_window attribute. With that, you have a sliding time window and can do a count by user and action (if that is what you need, otherwise you can just do a count by user). Docs here.

View solution in original post

Richfez
SplunkTrust
SplunkTrust

There are a handful of ways. In addition to jeffland's answer above, you could also...

... | transaction maxspan=1m User | search duration>X

That will group events with the same User (and not separated by more than 1 minute - adjust that) into a "set" of events. You then have a duration, eventcount, and other information.

More performance (especially in a distributed environment) can be had by converting that to a stats command.

... | stats max(_time) as Latest min(_time) as Earliest BY User | eval duration=Latest-Earliest | search duration>X

You'll like need a bit more list(fieldname) or 'values(fieldname)` in the stats command to get all your information out, but know it's possible. 🙂

Links to docs on stats and the functions stats supports, and finally transaction.

0 Karma

jeffland
SplunkTrust
SplunkTrust

streamstats has a time_window attribute. With that, you have a sliding time window and can do a count by user and action (if that is what you need, otherwise you can just do a count by user). Docs here.

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...