Monitoring Splunk

How to fetch the status of an application having multiple jobs ?

Real_captain
Path Finder

HI Team 

Can someone please help me to find how we can fetch the status of the application A1 having 5 jobs (Job1 , Job2 , Job3 , Job4 , Job5) running every day. 

Status of Application : This needs to be extracted using the query attached below: 
Planned : If current time is less than the expected time of JOB1 
OK-Running :  If Current time is between the expected time of JOB1 and expected time of JOB5 + Status of all the JOBs is either OK  or PLANNED
KO-FAILED : if Current time is between the expected time of JOB1 and expected time of JOB5 + Status of any the 1 JOBs is either KO. 

Query used today to fetch the status of each job in the application: 

index = ABC ( TERM(JOB1) OR TERM(JOB4) OR TERM(JOB2) OR TERM(JOB3) OR TERM(JOB5) OR TERM(JOB6) OR TERM(JOB7) ) ("- ENDED" OR "- STARTED" OR "ENDED - ABEND")
| eval Function = case(like(TEXT, "%ENDED - ABEND%"), "ABEND" , like(TEXT, "%ENDED - TIME%"), "ENDED" , like(TEXT, "%STARTED - TIME%"), "STARTED")
| eval DAT = strftime(relative_time(_time, "+0h"), "%d/%m/%Y") , {Function}_TIME=_time
| rename DAT as Date_of_reception
| stats max(Date_of_reception) as Date_of_reception max(ENDED_TIME) as ENDED_TIME max(STARTED_TIME) as STARTED_TIME max(ABEND_TIME) as ABEND_TIME by JOBNAME
| inputlookup append=t ESES_Job_MIFID_PPE.csv
| stats values(*) as * by JOBNAME
| eval DAY_OF_WEEK = strftime(strptime(Date_of_reception, "%d/%m/%Y"), "%A") ,today = strftime(1743030000, "%Y-%m-%d") , TO_DAY = strftime(strptime(today, "%Y-%m-%d"), "%A") , Diff=ENDED_TIME-STARTED_TIME
| rename STARTED_TIME as START_TIME1 , ENDED_TIME as END_TIME1 , ABEND_TIME as ABEND_TIME1
| eval diff_time = tostring(Diff , "duration"), diff_time_1=substr(diff_time,1,8) ,
START_TIME = Date_of_reception." ".strftime((START_TIME1),"%H:%M:%S") ,
END_TIME = Date_of_reception." ".strftime((END_TIME1),"%H:%M:%S") , END_TIME2 = strftime((END_TIME1),"%H:%M:%S") ,
ABEND_TIME = Date_of_reception." ".strftime((ABEND_TIME1),"%H:%M:%S") , ABEND_TIME2 = strftime((ABEND_TIME1),"%H:%M:%S") ,
EXPECTED_TIME = exp_time , DEADLINE_TIME = high_dl2
```EXPECTED_TIME_run = Date_of_reception." ".EXPECTED_TIME, EXPECTED_TIME_run = strptime(EXPECTED_TIME_run, "%d/%m/%Y %H:%M:%S") , TimeDiff=EXPECTED_TIME_run-now() ,
EXP_TIME_norun = if (TO_DAY = "Friday" , exp_time2 , exp_time1) ,EXPECTED_TIME_norun = today + " " + EXP_TIME_norun, EXPECTED_TIME_norun = strptime(EXPECTED_TIME_norun, "%Y-%m-%d %H:%M:%S") , TimeDiff_norun =EXPECTED_TIME_norun-now()
, Time_Diff=now() - strptime(START_TIME, "%d/%m/%Y %H:%M:%S") ```
| eval STATUS =
if(isnotnull(END_TIME2) AND (END_TIME2 <= ABEND_TIME2),"ABEND",
if(isnotnull(END_TIME2) AND (END_TIME2 <= DEADLINE_TIME),"OK",
if(isnotnull(END_TIME2) AND (END_TIME2 > DEADLINE_TIME),"BREACHED",
if(isnull(END_TIME2) AND isnull(START_TIME1) AND (TimeDiff_norun > 300),"PLANNED",
if(isnull(END_TIME2) AND isnull(START_TIME1) AND isnull(TimeDiff) AND (TimeDiff_norun < -600) AND (TimeDiff_norun >= -1800),"JOB NOT STARTED YET",
if(isnull(END_TIME2) AND isnull(START_TIME1) AND isnull(TimeDiff) AND (TimeDiff_norun < -1800),"JOB DID NOT EXECUTED",
if(isnull(END_TIME2) AND isnotnull(START_TIME1) AND (Time_Diff>600),"FAILED",
if(isnull(END_TIME2) AND isnotnull(START_TIME1) and (TimeDiff<=600),"RUNNING",
if( isnull(END_TIME2) AND isnull(START_TIME1) AND JOBNAME IN ("$JOB3" ) , "OK-Interest file is received" ,
if( isnull(END_TIME2) AND isnull(START_TIME1) AND JOBNAME IN ("$JOB6") , "OK-Mifid 2 file is received" ,
if( isnotnull(END_TIME2) AND isnotnull(START_TIME1) AND JOBNAME IN ("$JOB3" ) , "KO-Interest file Not received" ,
if( isnotnull(END_TIME2) AND isnotnull(START_TIME1) AND JOBNAME IN ("$JOB6") , "KO-Mifid 2 file Not received" ,
"WARNING"))))))))))))
| rename diff_time_1 as EXECUTION_TIME
| sort Order
| table Application,JOBNAME,Description, EXPECTED_TIME , DEADLINE_TIME , START_TIME , END_TIME ,EXECUTION_TIME, STATUS
| fillnull value="-"

 

Labels (1)
0 Karma

Real_captain
Path Finder

@ITWhisperer  

Can you please help me in this topic. 

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Please do not tag me - I, like many here, volunteer my time and expertise and it is not for others to suggest what I work on. By specifically addressing people, you are also potentially excluding others who may have valuable contributions to make; it is like you don't value or are not interested in their efforts (since you haven't also directly addressed them). I imagine this can be counter-productive to resolving your issue!

Real_captain
Path Finder

@isoutamo 
is it possible to correct my splunk query to fetch the status of the application as below : 

Status of Application : This needs to be extracted using the query attached below: 
Planned : If current time is less than the expected time of JOB1 
OK-Running :  If Current time is between the expected time of JOB1 and expected time of JOB5 + Status of all the JOBs is either OK  or PLANNED
KO-FAILED : if Current time is between the expected time of JOB1 and expected time of JOB5 + Status of any the 1 JOBs is either KO. 

0 Karma

Real_captain
Path Finder

Sample events are attached. 

 

 
 

 

 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Have you any sample events which you can share with us? If yes, then anonymous those if/as needed.
0 Karma

Real_captain
Path Finder

@isoutamo  or Someone from Splunk Team 

Can you please help to provide me a solution for this type of result. 

 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Your Sample logs.txt didn't work with your query. I suppose that it's not a real sample data what you have gotten from index=.... query!
Also there is one lookup which content is unknown.
0 Karma

Real_captain
Path Finder

Splunk query + Csv file + real output data is attached. 

Real_captain_0-1743666043902.png

 

 

0 Karma

Real_captain
Path Finder

Csv file is attached 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
We don’t need an output files. Raw input events are what’s still missing.
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Sample events have been provided below, but, unfortunately, they don't match the supplied lookup and are not in a good format (fields and headers have different delimiters and are consequently not aligned well!)

0 Karma

isoutamo
SplunkTrust
SplunkTrust
I already try those after fixing delimiters etc, but SPL expects some _time which aren’t present in this example file.
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...