Splunk Search

transaction endswith usage

sethuk555
Engager

Hi,

We are trying to calculate the total system down time in minutes in a tabular format using the below query

sourcetype="nodecheck" | rex "System (?<upordown>.*)" | dedup upordown consecutive=true sortby +_time
| transaction startswith="System Up" endswith="System Down*" | EVAL Outage(Minutes)=ceiling(duration/60)
| TABLE _time Outage(Minutes) | rename _time as DownTimeStart | convert ctime(DownTimeStart)
| addcoltotals labelfield=Outage(Minutes) | fillnull value="Total"

But, somehow we are unable to combine these two types of system down logs "System Down" & "System Down - Unable to connect to the Database!" in our total calculation of outage. It somehow just takes the "System Down" into consideration. Below is the data set.
we are unable to use OR clause in the endswith utility under transaction.

2014-03-17 15:42:01-System Up
2014-03-17 15:37:01-System Up
2014-03-17 15:32:41-System Down
2014-03-17 15:25:01-System Down - Unable to connect to the Database!
2014-03-17 15:15:01-System Down - Unable to connect to the Database!
2014-03-17 15:05:02-System Down - Unable to connect to the Database!
2014-03-17 14:52:01-System Up
2014-03-17 14:47:01-System Up
2014-03-17 14:42:01-System Up

Tags (2)
0 Karma

lguinn2
Legend

I see several problems:

First, ( is not a valid character in a field name - unless you use quotation marks and then things get quite complicated.

Second, Splunk orders events, and transaction makes some assumptions about that order. Your dedup command was changing the default order. I wouldn't do that.

Third, I simplified and combined a few steps.

Try this:

sourcetype="nodecheck" 
| rex "System (?<upordown>(?:Up|Down))" 
| dedup upordown consecutive=true
| transaction startswith="System Down" endswith="System Up" 
| eval Outage_Minutes=ceiling(duration/60) 
| eval DownTimeStart = strftime(_time,"%x %X")
| table DownTimeStart Outage_Minutes 
| addcoltotals labelfield=Outage_Minutes label="Total"
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Monitoring AI Agents with Splunk Observability Cloud

Let’s say I’m running a travel planning AI app in production. A user asks for three concise hotel options in ...

[Puzzles] Solve, Learn, Repeat: Tiling

This puzzle (first published here) is based on finding groups of tessellated tiles (inspired by floor tiles I ...

SOK it to Me: Top 3 Benefits of Using Splunk Operator on Kubernetes that’ll Make ...

    Thursday, July 9, 2026  |  11:00AM–12:00PM PDT Duration: 1 hour (includes Q&A) Managing can feel like a ...