monitor failed jobs

[sarit_s]

Hello

i want to create a dashboard that monitors the failed job (savedsearches the i can see in the activity page)

how can search for failed jobs ? 
i saw that ITSI can do it but i wonder if there is a way to do it with Splunk itself

thanks

[inventsekar]

Hi Sarit, Please check these search queries:

| rest /services/search/jobs
| table dispatchState *.search

[sarit_s]

the first one looks close to what i want. is there a way to filter so i will see only failed jobs ?

[inventsekar]

| rest /services/search/jobs isFailed=1

this will list only failed jobs. 

if this solved your query, please accept this as the solution.  

[sarit_s]

well, there is something weird

even thought im searching for isFailed=1 im getting results of isFailed=0

image below

[inventsekar · Accepted solution]

Check...

 | rest /services/search/jobs | where isFailed=1

Or

| rest /services/search/jobs | search isFailed=1

PS - Karma points are appreciated, if a reply solved your problem, please accept it as the solution. 

[inventsekar]

i assumed that "failed jobs" meaning, some app related failed jobs. 

for splunk's failed jobs, 

Splunk GUI---->Activity---> Jobs---> Status dropdown, select Failed.

or, the page URL is(pls update your splunk link on this below URL)

https://yourcompany.splunk.com/en-US/app/search/job_manager?owner=&jobStatus=failed

[sarit_s]

I know it is there

i want to monitor this jobs and add it as dashboard panel

is there a query that can give me the results ?

[inventsekar]

Hi @sarit_s by Splunk also, its possible. 

the applications logs are already ingested to splunk, we hope. and if you are not sure how to search for the failed jobs, then, maybe, you start from the hostname. simply search for the host which is running the app. 

then, from the events list, you can find out the source/sourcetype. you may need to do field extractions using the rex command. 

let us know how your search goes, so that we can help you. thanks. 

[sarit_s]

In the jobs list i see the inpect job option. I see there field name dispatchedStatus but i cant search for it

is there a way to search something like this ?