Splunk Search

How does Splunk handle transactions that span search time boundaries?

cantgetnosleep
Explorer

How does splunk handle transactions that span search time boundaries? If a transaction starts before a search interval, but finishes within it, is it included in the search? Also, if a transaction begins within the search interval but ends after it, how is that handled?

Thanks,

Andrew

Tags (3)
1 Solution

MuS
SplunkTrust
SplunkTrust

Hi cantgetnosleep,

here is what doc tells us about transaction:

 A transaction search enables you to identify transaction events that each stretch over multiple logged events. 

that said, if you search from 8am til 10am the transaction command will use the events returned from your base search and builds a group of conceptually-related events. Since your base search only returns events from 8am til 10am, the transaction command will not know if there is anything before 8am or after 10am.

hope this helps ...

cheers, MuS

View solution in original post

MuS
SplunkTrust
SplunkTrust

Hi cantgetnosleep,

here is what doc tells us about transaction:

 A transaction search enables you to identify transaction events that each stretch over multiple logged events. 

that said, if you search from 8am til 10am the transaction command will use the events returned from your base search and builds a group of conceptually-related events. Since your base search only returns events from 8am til 10am, the transaction command will not know if there is anything before 8am or after 10am.

hope this helps ...

cheers, MuS

cantgetnosleep
Explorer

Awesome. Thanks! Those were very helpful answers.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Transactions that entirely are in the third hour will be searched in the next scheduled execution. Transactions that had their "tail" in the first hour were already found by the previous scheduled execution.

martin_mueller
SplunkTrust
SplunkTrust

You schedule a search for 2 */2 * * * or however long you want to wait for events to be present from distant systems, set the time range to -3h@h to @h, and build your search like this:

base search | transaction blah blah | search to make sure a transaction is complete | addinfo | where _time < relative_time(info_min_time, "+2h")

The last where is key. You search over three hours, assemble transactions, and then only keep transactions that started in the first two hours. As a result you get transactions whose "tail" is in the third hour.

martin_mueller
SplunkTrust
SplunkTrust

In other words, it doesn't handle that.

Whether these "half transactions" are included in your results depends on the configuration of the transaction command and the available data. For example, if your transaction only has a start and end event then I'd say seeing only one of those would cause that transaction to be evicted and hidden unless explicitly shown with keepevicted=t.

To get around that you should always run overlapping transaction searches. Say you know a transaction is at most one hour long and you want to schedule a search every two hours over the previous two hours.

Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...