Splunk Search

Group Events Based Solely on Temporal Proximity

fervin
Path Finder

Hi all,

We have a need to correlate IPS, application, and firewall logs based solely on their timestamps.

The reason is because our proxy servers and various firewalls perform Network Address Translation (NAT) and thus events don't share common src_ip or dst_ip in all cases.

I'm familiar with transactions, and they are great, but only when each event involved in the correlation shares a common field value. What we're looking for is a way to correlate based on a specified (likely very small) span of time between events of two or more sourcetypes.

Anyone have a good way of doing this? Thanks in advance,

-Frank

Tags (2)
0 Karma
1 Solution

Stephen_Sorkin
Splunk Employee
Splunk Employee

I would use transaction as gkanapathy suggests in his answer. You can then post-filter those transactions to only include those with BOTH sourcetypes. For example:

sourcetype=ips OR sourcetype=*syslog* | transaction maxpause=1s | search sourcetype=ips sourcetype=*syslog*

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee

I would use transaction as gkanapathy suggests in his answer. You can then post-filter those transactions to only include those with BOTH sourcetypes. For example:

sourcetype=ips OR sourcetype=*syslog* | transaction maxpause=1s | search sourcetype=ips sourcetype=*syslog*

hulahoop
Splunk Employee
Splunk Employee
0 Karma

gkanapathy
Splunk Employee
Splunk Employee

As a matter of fact transaction does not require items to share any common fields. You can just specify maxspan (time from first event) and/or maxpause (max time between events), and all events piped to it will be grouped by those parameters.

I suppose I should note that transaction is mostlyuseful for finding out and grouping events in aggregate. If you're looking at a specific set of events in a specific time range, it is much more efficient specify your time range (or use a subsearch to generate a time range around a specific event) and use the search terms to get the sourcetypes you're interested in seeing.

0 Karma

fervin
Path Finder

Thanks for the prompt response. So, you're right about transaction not requiring common fields, but I can't see how to force it to give me what I need for this use case. It's very likely that I'm missing something incredibly simple...

An example:

sourcetype=ips OR sourcetype=syslog | transaction maxpause=1s

This could potentially give me bundles of all syslog, with no ips events.

What I want would be the functional equivalent of something like this fake query:

sourcetype=syslog | near sourcetype=ips maxpause=1s

Does this make sense?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...