How can I remove events that are repeated consecutively?
For example, my logs shows:
Timestamp 1 | Event A | User 1
Timestamp 2 | Event B | User 2
Timestamp 3 | Event B | User 2
Timestamp 4 | Event C | User 1
Timestamp 5 | Event C | User 1
Timestamp 6 | Event C | User 2
Timestamp 7 | Event A | User 2
How can I make it to:
Timestamp 1 | Event A | User 1
Timestamp 2 | Event B | User 2
Timestamp 4 | Event C | User 1
Timestamp 6 | Event C | User 2
Timestamp 7 | Event A | User 2
I am able to cluster events that occur close to each other via transaction maxpause
, but it also clusters events from different users. How can I get around it?
Thank you!
You can specify fields when using transaction aswell as the maximum time that the events can be apart. That should un-clutter your transactions.
your search | transaction User maxspan=5m
Here's run-anywhere code to make your test data...
| makeresults | eval mydata= "1=A 2=B 2=B 1=C 1=C 2=C 2=A" | makemv mydata | mvexpand mydata | rex field=mydata "(?<User>[^=]+)=(?<Event>.*)" | eval User = "User ".User | eval Event="Event ".Event | streamstats count as recno | eval _time =relative_time(now(),"-1h@h") + recno
| rename COMMENT as "The above just generates your test data."
Then you can do either this....
| rename COMMENT as "This assumes that the Events have a completely unique key."
| stats min(_time) as _time by User Event | table _time Event User | sort 0 _time
... or this...
| rename COMMENT as "This assumes that the Events have an event type that is not unique."
| sort 0 _time
| streamstats current=f last(Event) as lastEvent by User
| where isnull(lastEvent) OR Event!=lastEvent
You can specify fields when using transaction aswell as the maximum time that the events can be apart. That should un-clutter your transactions.
your search | transaction User maxspan=5m