Splunk Search

Group Events Based Solely on Temporal Proximity

fervin
Path Finder

Hi all,

We have a need to correlate IPS, application, and firewall logs based solely on their timestamps.

The reason is because our proxy servers and various firewalls perform Network Address Translation (NAT) and thus events don't share common src_ip or dst_ip in all cases.

I'm familiar with transactions, and they are great, but only when each event involved in the correlation shares a common field value. What we're looking for is a way to correlate based on a specified (likely very small) span of time between events of two or more sourcetypes.

Anyone have a good way of doing this? Thanks in advance,

-Frank

Tags (2)
0 Karma
1 Solution

Stephen_Sorkin
Splunk Employee
Splunk Employee

I would use transaction as gkanapathy suggests in his answer. You can then post-filter those transactions to only include those with BOTH sourcetypes. For example:

sourcetype=ips OR sourcetype=*syslog* | transaction maxpause=1s | search sourcetype=ips sourcetype=*syslog*

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee

I would use transaction as gkanapathy suggests in his answer. You can then post-filter those transactions to only include those with BOTH sourcetypes. For example:

sourcetype=ips OR sourcetype=*syslog* | transaction maxpause=1s | search sourcetype=ips sourcetype=*syslog*

hulahoop
Splunk Employee
Splunk Employee
0 Karma

gkanapathy
Splunk Employee
Splunk Employee

As a matter of fact transaction does not require items to share any common fields. You can just specify maxspan (time from first event) and/or maxpause (max time between events), and all events piped to it will be grouped by those parameters.

I suppose I should note that transaction is mostlyuseful for finding out and grouping events in aggregate. If you're looking at a specific set of events in a specific time range, it is much more efficient specify your time range (or use a subsearch to generate a time range around a specific event) and use the search terms to get the sourcetypes you're interested in seeing.

0 Karma

fervin
Path Finder

Thanks for the prompt response. So, you're right about transaction not requiring common fields, but I can't see how to force it to give me what I need for this use case. It's very likely that I'm missing something incredibly simple...

An example:

sourcetype=ips OR sourcetype=syslog | transaction maxpause=1s

This could potentially give me bundles of all syslog, with no ips events.

What I want would be the functional equivalent of something like this fake query:

sourcetype=syslog | near sourcetype=ips maxpause=1s

Does this make sense?

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...