Splunk Search

How to remove consecutive repeat events by an unique user?

tanyongjin
Explorer

How can I remove events that are repeated consecutively?

For example, my logs shows:
Timestamp 1 | Event A | User 1
Timestamp 2 | Event B | User 2
Timestamp 3 | Event B | User 2
Timestamp 4 | Event C | User 1
Timestamp 5 | Event C | User 1
Timestamp 6 | Event C | User 2
Timestamp 7 | Event A | User 2

How can I make it to:
Timestamp 1 | Event A | User 1
Timestamp 2 | Event B | User 2
Timestamp 4 | Event C | User 1
Timestamp 6 | Event C | User 2
Timestamp 7 | Event A | User 2

I am able to cluster events that occur close to each other via transaction maxpause, but it also clusters events from different users. How can I get around it?

Thank you!

Tags (2)
0 Karma
1 Solution

ckunath
Communicator

You can specify fields when using transaction aswell as the maximum time that the events can be apart. That should un-clutter your transactions.

your search | transaction User maxspan=5m

View solution in original post

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Here's run-anywhere code to make your test data...

| makeresults | eval mydata= "1=A 2=B 2=B 1=C 1=C 2=C 2=A" | makemv mydata | mvexpand mydata | rex field=mydata "(?<User>[^=]+)=(?<Event>.*)" | eval User = "User ".User | eval Event="Event ".Event | streamstats count as recno | eval _time =relative_time(now(),"-1h@h") + recno 
| rename COMMENT as "The above just generates your test data."

Then you can do either this....

| rename COMMENT as "This assumes that the Events have a completely unique key."
| stats min(_time) as _time by User Event | table _time Event User  | sort 0 _time

... or this...

| rename COMMENT as "This assumes that the Events have an event type that is not unique."
| sort 0 _time 
| streamstats current=f last(Event) as lastEvent by User
| where isnull(lastEvent) OR Event!=lastEvent
0 Karma

ckunath
Communicator

You can specify fields when using transaction aswell as the maximum time that the events can be apart. That should un-clutter your transactions.

your search | transaction User maxspan=5m
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...