I'd like to monitor and alert on the number of files in the dispatch directory.. What's the best way to accomplish this?
You could build an alert using the following query:
| rest /services/search/jobs | stats count | where count>2000
Another option: I setup a monitor / report that breaks down both disk usage and dispatch jobs by user (as well as provides total amounts) so that I know who to talk to about anything crazy I see. Includes percentages as well.
| rest /services/search/jobs
| eval diskUsageMB=diskUsage/1024/1024
| rename eai:acl.owner as user
| eventstats count AS Total_Jobs, sum(diskUsageMB) AS Total_Storage
| eventstats count AS Jobs, sum(diskUsageMB) AS Storage by user
| eval Job%=round((Jobs/Total_Jobs)*100,1) . "%"
| eval Storage%=round((Storage/Total_Storage)*100,1) . "%"
| dedup user
| table user Jobs Total_Jobs Job% Storage Total_Storage Storage%
| sort - Jobs
This allows you to set both per user and overall alerts on either (or both) disk usage and count of jobs. Just add a
| where Jobs > X OR Total_Jobs > Y AND Storage > Z, etc.
only thing is that if you want to do math on the % fields, you'll want to either convert to number or remove the "%" sign I added in the evals.
You could build an alert using the following query:
| rest /services/search/jobs | stats count | where count>2000
Perfect, Thanks!