Reporting

Why our hourly report doesn't show up?

ddrillic
Ultra Champion

We have an hourly report that runs manually just fine but the scheduled run doesn't seem to work. Where do we look?

Some reports of this user work and some don't.

Tags (2)
0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @ddrilic,

Can you please check in $SPLUNK_HOME/var/log/splunk/scheduler.log ? You can try to run this query index=_internal host=<SEARCHHEAD> source=*scheduler.log savedsearch_name=<SCHEDULED_SEARCHNAME> on search head and then you can check whether it triggered at scheduled time or not.

0 Karma

ddrillic
Ultra Champion

Great @harsmarvania57.

I ran index=_internal source=*scheduler.log savedsearch_name=<report name>

It gives an error message at the scheduled time -

status=delegated_remote_error, error accessing https://<SH 2>:8089/servicesNS/<user id>/<app name>/shcluster/member/<report name>/sched_dispatch?output_mode=json, statusCode=404, description=Not Found
0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Which version of splunk are you running ? Because I have checked on 6.4.X and 6.6.X SH cluster and I can't find this URL https://<SH >:8089/servicesNS/<user id>/<app name>/shcluster/member/<report name> , till member URL is exist but after that other options are available not <report name> so I would suggest you to raise case with splunk.

0 Karma

ddrillic
Ultra Champion

Very interesting @harsmarvania57 - we are at 6.5.2. A case with Support was opened...

0 Karma

ddrillic
Ultra Champion

Based on Search Head Cluster scheduled searches: What are these Status values in scheduler.log?

I ran -

index=_internal sourcetype=*sched* source="*scheduler.log" | timechart span=1day count by status

It shows -
delegated_remote - 1782
delegated_remote_completion - 1781
delegated_remote_error - 415

0 Karma

lim2
Path Finder

We experiencing quite a few of status=delegated_remote_error and status=skipped searches on Splunk SHC ver 8.0.2

even with following setting:

Total Scheduled searches = (((base_max_searches + cpu_count*max_search_per_cpu) * max_searches_perc) / 100) * num_members
Total schedule searches = ((6 + 36*1) * 50/100) * 7 = 21 * 7 = 147

Is there other setting tuning variables?

Thanks.

0 Karma

somesoni2
Revered Legend

Now was the alert created? On the search head (see host field) where you see status=delegated_remote_error, does it exists or have correct permissions?

0 Karma

ddrillic
Ultra Champion

On the search head that the error refers to, the alert exists. What should the permissions be?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!