Splunk Search
Highlighted

Only return single event that has reoccurring field in 1 minute span.

Path Finder

I am trying to create a search that would return results through stats. I have a field called srcip and I only want to see events if there is a match of the srcip against other events but only matching within a minute window. The logic is I only want to see one event if there is more than one matching srcip specific to the user in the event within the given 1 minute window. So I tried the dedup command but was not sure if I could dedup the srcip field within the span of a minute using either the time field provided by splunk or use a begintime field that comes from the sourcetype.

| transaction srcip maxpause=60s

That groups events by the field src
ip and also groups events that are no more than 60 seconds apart. This might work if I could output a single event from the group. But I also don't want to see events that only occur once and dont reoccur more than once per 60 seconds.

0 Karma
Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

SplunkTrust
SplunkTrust

Have you looked at transaction command? It provides options to group events based on various criteria and one of them maxspan.
http://docs.splunk.com/Documentation/Splunk/6.3.0/SearchReference/Transaction

0 Karma
Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

Path Finder

So I tried transaction maxspan=1m and i got 11 events back but almost 9,000 lines of the same events.

0 Karma
Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

SplunkTrust
SplunkTrust

did you include field src_ip in the transaction , like this?

your base search | transaction maxspan=1m src_ip
0 Karma
Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

Path Finder

Tried it and that transaction command seems to only group up events. It does not filter it out. Also it seems to duplicate events I have because out of the 100 or so results I have There are over 9k rows.

0 Karma
Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

SplunkTrust
SplunkTrust

Can you post some sample data?

0 Karma
Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

Esteemed Legend

Like this:

... | bucket _time span=1m | dedup _time src_ip

Or like this:

... | bucket _time span=1m | stats first(_raw) AS LatestEventThisMinute by src_ip

View solution in original post

Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

Path Finder

The dedup works. Is there a way to do a count against the total ammount of events found within that 60 second window that was deduped?

0 Karma
Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

Esteemed Legend

Like this:

 ... | bucket _time span=1m | stats count latest(_raw) AS LatestEventThisMinute by src_ip
0 Karma
Highlighted

Re: Only return single event that has reoccurring field in 1 minute span.

Path Finder

This is the query im using
index=symantec sourcetype=sep:ids
| bucket time span=1m | rename destip as LocalHostIP | rename srcip as RemoteHostIP
| stats count latest(
raw) AS LatestEventThisMinute by
RemoteHostIP,_time | where count > 1

I want to pull out other fields for each grouped event. Since they have the same src ip each event will be pretty much the same in terms of fields. Is it possible to stats by other fields without parsing against them to see if they match the criteria of being the same?

0 Karma