Splunk Search

How to edit my search to find orphaned transactions older than 1 minute?

recurse
New Member

Hello. I have a search that looks for orphaned transactions, as follows:

[...main search...]
| transaction request_id keeporphans=true maxspan=1m 
| search _txn_orphan=1

It currently works, but sometimes yields false negatives if a transaction happens to be in the middle of processing within the last minute. I'm trying to amend my search to only return orphans that occurred at least a minute ago, but am not having any luck:

[...main search...]
| transaction request_id keeporphans=true maxspan=1m 
| search _txn_orphan=1 latest=-1m

Anyone have any ideas on how to accomplish this? I've tried several variations of date math without any luck.

0 Karma
1 Solution

somesoni2
Revered Legend

Try like this

Updated comparison operator of where clause

[...main search...]
 | transaction request_id keeporphans=true maxspan=1m 
 | where _txn_orphan=1 AND _time<relative_time(now(),"-1m@m")

View solution in original post

0 Karma

somesoni2
Revered Legend

Try like this

Updated comparison operator of where clause

[...main search...]
 | transaction request_id keeporphans=true maxspan=1m 
 | where _txn_orphan=1 AND _time<relative_time(now(),"-1m@m")
0 Karma

recurse
New Member

Thanks for the reply - I hadn't tried a variation with the '@m' suffix. Unfortunately, it excludes all of my results, including those which are older than 1m ago. Baffling.

0 Karma

recurse
New Member

Also, I switched > to < to align with what I'm looking for, still no luck.

0 Karma

somesoni2
Revered Legend

The comparison operator should've been <, updated the same. Try with just -1m instead of -1m@m. Let us know if it doesn't work and what the problem is?

0 Karma

recurse
New Member

Still no luck. I find 4 orphaned transactions (from over an hour ago) without the additional constraint and none with it. Let me know if there's any additional information I can provide.

0 Karma

somesoni2
Revered Legend

I guess we need more information on how things are logged (with samples) in your data. How can we differentiate between an orphan and in-progress transaction? I believe that's where you're getting false +ve and that's what we need to fix.

0 Karma

recurse
New Member

Oh, I missed your addition of the 'where' clause in the edit. I added that and it's working great. Thank you!

0 Karma
Get Updates on the Splunk Community!

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...