Getting Data In

How to use time elapsed after an event and a missing event to create an alert?

alexm2a
Engager

Hi there,

 

I have 2 messages that log when a job is run, which share a job_id field 

event_name=process.start 

event_name=process.end

 

I'm trying to create an alert that fires if there is an event_name=process.start , but no event_name=process.end , after 3 hours.

I've seen lots of examples of using transactions between 2 events to get the duration, but not any if an event is missing.

Many thanks, apologies if this is a noob question

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @alexm2a,

please try something like this:

index=your_index (event_name="process.start" OR event_name="process.end")
| transaction maxspan=3h startswith=event_name=process.start endswith=event_name=process.end
| where eventcount=1 AND event_name="process.start"

Ciao.

Giuseppe

alexm2a
Engager

Hi Giuseppe, thank you for your assistance.

 

When I try that I get no results even though I can see a job more than 4 hours ago that had a start but no end has happened.  

If I remove the

| where eventcount=1 AND event_name="process.start"


and use 


| table event_name, jobId 

 
I cannot see my stuck jobId in the list

Seems like the transaction is removing the event that does not have a process.end 

Any ideas? 

 

Many thanks

Alex

0 Karma
Get Updates on the Splunk Community!

Pro Tips for First-Time .conf Attendees: Advice from SplunkTrust

Heading to your first .Conf? You’re in for an unforgettable ride — learning, networking, swag collecting, ...

Raise Your Skills at the .conf25 Builder Bar: Your Splunk Developer Destination

Calling all Splunk developers, custom SPL builders, dashboarders, and Splunkbase app creators – the Builder ...

Hunt Smarter, Not Harder: Discover New SPL “Recipes” in Our Threat Hunting Webinar

Are you ready to take your threat hunting skills to the next level? As Splunk community members, you know the ...