Getting Data In

How to use time elapsed after an event and a missing event to create an alert?

alexm2a
Engager

Hi there,

 

I have 2 messages that log when a job is run, which share a job_id field 

event_name=process.start 

event_name=process.end

 

I'm trying to create an alert that fires if there is an event_name=process.start , but no event_name=process.end , after 3 hours.

I've seen lots of examples of using transactions between 2 events to get the duration, but not any if an event is missing.

Many thanks, apologies if this is a noob question

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @alexm2a,

please try something like this:

index=your_index (event_name="process.start" OR event_name="process.end")
| transaction maxspan=3h startswith=event_name=process.start endswith=event_name=process.end
| where eventcount=1 AND event_name="process.start"

Ciao.

Giuseppe

alexm2a
Engager

Hi Giuseppe, thank you for your assistance.

 

When I try that I get no results even though I can see a job more than 4 hours ago that had a start but no end has happened.  

If I remove the

| where eventcount=1 AND event_name="process.start"


and use 


| table event_name, jobId 

 
I cannot see my stuck jobId in the list

Seems like the transaction is removing the event that does not have a process.end 

Any ideas? 

 

Many thanks

Alex

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...