Getting Data In

How to use time elapsed after an event and a missing event to create an alert?

alexm2a
Engager

Hi there,

 

I have 2 messages that log when a job is run, which share a job_id field 

event_name=process.start 

event_name=process.end

 

I'm trying to create an alert that fires if there is an event_name=process.start , but no event_name=process.end , after 3 hours.

I've seen lots of examples of using transactions between 2 events to get the duration, but not any if an event is missing.

Many thanks, apologies if this is a noob question

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @alexm2a,

please try something like this:

index=your_index (event_name="process.start" OR event_name="process.end")
| transaction maxspan=3h startswith=event_name=process.start endswith=event_name=process.end
| where eventcount=1 AND event_name="process.start"

Ciao.

Giuseppe

alexm2a
Engager

Hi Giuseppe, thank you for your assistance.

 

When I try that I get no results even though I can see a job more than 4 hours ago that had a start but no end has happened.  

If I remove the

| where eventcount=1 AND event_name="process.start"


and use 


| table event_name, jobId 

 
I cannot see my stuck jobId in the list

Seems like the transaction is removing the event that does not have a process.end 

Any ideas? 

 

Many thanks

Alex

0 Karma
Get Updates on the Splunk Community!

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...

Splunk MCP & Agentic AI: Machine Data Without Limits

Discover how the Splunk Model Context Protocol (MCP) Server can revolutionize the way your organization uses ...