I am trying to create a search that would return results through stats. I have a field called srcip and I only want to see events if there is a match of the srcip against other events but only matching within a minute window. The logic is I only want to see one event if there is more than one matching srcip specific to the user in the event within the given 1 minute window. So I tried the dedup command but was not sure if I could dedup the srcip field within the span of a minute using either the time field provided by splunk or use a begintime field that comes from the sourcetype.
| transaction srcip maxpause=60s
That groups events by the field srcip and also groups events that are no more than 60 seconds apart. This might work if I could output a single event from the group. But I also don't want to see events that only occur once and dont reoccur more than once per 60 seconds.
Have you looked at transaction command? It provides options to group events based on various criteria and one of them maxspan.
So I tried transaction maxspan=1m and i got 11 events back but almost 9,000 lines of the same events.
did you include field src_ip in the transaction , like this?
your base search | transaction maxspan=1m src_ip
Tried it and that transaction command seems to only group up events. It does not filter it out. Also it seems to duplicate events I have because out of the 100 or so results I have There are over 9k rows.
... | bucket _time span=1m | dedup _time src_ip
Or like this:
... | bucket _time span=1m | stats first(_raw) AS LatestEventThisMinute by src_ip
The dedup works. Is there a way to do a count against the total ammount of events found within that 60 second window that was deduped?
... | bucket _time span=1m | stats count latest(_raw) AS LatestEventThisMinute by src_ip
This is the query im using
| bucket time span=1m | rename destip as LocalHostIP | rename srcip as RemoteHostIP
| stats count latest(raw) AS LatestEventThisMinute by
RemoteHostIP,_time | where count > 1
I want to pull out other fields for each grouped event. Since they have the same src ip each event will be pretty much the same in terms of fields. Is it possible to stats by other fields without parsing against them to see if they match the criteria of being the same?