Only return single event that has reoccurring field in 1 minute span.

[santorof]

I am trying to create a search that would return results through stats. I have a field called src_ip and I only want to see events if there is a match of the src_ip against other events but only matching within a minute window. The logic is I only want to see one event if there is more than one matching src_ip specific to the user in the event within the given 1 minute window. So I tried the dedup command but was not sure if I could dedup the src_ip field within the span of a minute using either the _time field provided by splunk or use a begin_time field that comes from the sourcetype.

| transaction src_ip maxpause=60s

That groups events by the field src_ip and also groups events that are no more than 60 seconds apart. This might work if I could output a single event from the group. But I also don't want to see events that only occur once and dont reoccur more than once per 60 seconds.

[woodcock · Accepted solution]

Like this:

... | bucket _time span=1m | dedup _time src_ip

Or like this:

... | bucket _time span=1m | stats first(_raw) AS LatestEventThisMinute by src_ip

[santorof]

The dedup works. Is there a way to do a count against the total ammount of events found within that 60 second window that was deduped?

[woodcock]

Like this:

 ... | bucket _time span=1m | stats count latest(_raw) AS LatestEventThisMinute by src_ip

[santorof]

This is the query im using
index=symantec sourcetype=sep:ids
| bucket _time span=1m | rename dest_ip as Local_Host_IP | rename src_ip as Remote_Host_IP
| stats count latest(_raw) AS LatestEventThisMinute by
Remote_Host_IP,_time | where count > 1

I want to pull out other fields for each grouped event. Since they have the same src ip each event will be pretty much the same in terms of fields. Is it possible to stats by other fields without parsing against them to see if they match the criteria of being the same?

[woodcock]

You can use eventstats instead of stats and adjust accordingly:

http://docs.splunk.com/Documentation/Splunk/6.3.0/SearchReference/Eventstats

[somesoni2]

Have you looked at transaction command? It provides options to group events based on various criteria and one of them maxspan.
http://docs.splunk.com/Documentation/Splunk/6.3.0/SearchReference/Transaction

[santorof]

So I tried transaction maxspan=1m and i got 11 events back but almost 9,000 lines of the same events.

[somesoni2]

did you include field src_ip in the transaction , like this?

your base search | transaction maxspan=1m src_ip

[santorof]

Tried it and that transaction command seems to only group up events. It does not filter it out. Also it seems to duplicate events I have because out of the 100 or so results I have There are over 9k rows.

[somesoni2]

Can you post some sample data?