Which of my two searches is more accurate for monitoring scheduled searches?

[maurelio79]

Hi to all,

I'd like to know the difference between two kind of results that I get with 2 different searches:

1)

index=_internal sourcetype=scheduler host=hostA OR host=hostB savedsearch_name!=_Scheduled* |  stats max(run_time) by savedsearch_name, host |  rename max(run_time) AS runtime |  sort - runtime | head 10

This search returns a table with first 10 searches by runtime.

2)

index=_internal host=hostA OR host=hostB source=*scheduler.log |eval JOB_DELAY_SECS=(dispatch_time-scheduled_time)|search JOB_DELAY_SECS > 30 | eval pool=host +"_"+savedsearch_name | timechart span=1m perc95(JOB_DELAY_SECS) by pool useother=f limit=20

This search returns a graph with the difference between dispatch_time and scheduled_time, but this difference is not the runtime, am I right?

Which one of these searches is more correct to show most long running searches and/or most resource usage?

Thanks and regards.

[renjith_nair · Accepted solution]

Your first search gives you run time and your second search gives you the delay in execution; ie; if you have scheduled a search at 9:00 and executed at 9:05, then the delay is 5 minutes.

[maurelio79]

Sorry, last question:
To determinate the time range for the scheduled search, Splunk use scheduled time or dispatch time (in case there are relative time range, like -1m@m now)?

Thanks and regards.

[renjith_nair]

Normally it takes the scheduled time but Splunk considers different methods to run scheduled reports. http://docs.splunk.com/Documentation/Splunk/6.3.2/Report/Configurethepriorityofscheduledreports