Reporting

How to trigger alert if an event is not received by a certain time?

vgoli
Loves-to-Learn Lots

Hello, 

Need help with setting alerts for any event not started by a specific time.

I have a lookup file with details for many batch jobs with details like job name, Run date, Start time and alert time.

job name           run date                                                                     Start time     alert time

job1                      1st working date of every month                     9:00                  10:00

job 2                     1st working date of every month                    11:00                 12:00

job3                      1st working date of every month                     12:00                13:00

job4                      1st working date of every month                    13:00                 14:00

when the job runs we see an event with job name, start date, start time, end date, end time and status etc.

if any job is not started by the alert time on first of every month  I want to trigger an alert to notify the user that the batch job is not started.

Appreciate your help.

Vijay

 

Labels (1)
0 Karma

rrovers
Contributor

Is it an option to make an alert that checks the _internal index at the alert time if the job has run. If result is 0 then send an email.

0 Karma

vgoli
Loves-to-Learn Lots

Thanks for the quick response.

How to schedule the search for job1, in the lookup file, to run on first working day of every month at 10:00 ( search time in look up file)  to see if job has already started or not. Similarly for job 2 at 11:00 etc. 

 

0 Karma

rrovers
Contributor

I'm not sure if I understand your problem exactly. 

I think it's not possible to schedule a cron job only on working days. 
You could run a job on every first 3 days of the month and check in your search if this is a working day.

| makeresults 
| eval day=strftime(_time,"%a") 
| eval aantal=if(day="Mon" OR day ="Tue" OR day="Wed" OR day="Thu" OR day="Fri","0","1") 
| append 
    [ search index=_internal + your search criteria to find if the job has run
    | stats count AS aantal] 
| stats sum(aantal)

 In the alert you could set the option to send an email if number of events is 0.

If this doesn't solve your problem please give some more details (I don't understand the function of your lookup file, is that just an overview or do you want to run from that lookup or something?, the latter is not possible I think).

0 Karma

vgoli
Loves-to-Learn Lots

I have many different jobs in the backend system and each job runs monthly once on first working day at a fixed time, but this fixed time is different for each job( job1 starts at 9, job2 starts at 9:10, job3 starts at 11 and so on). The start time information of each job is available in the lookup file.

when the job starts we receive an event with necessary data

my requirement is, if any job is not started at its specified start time, I have to notify the job owner to inform that the job is not started so he can look into it.

 

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...