Splunk Search

Splunk Search Head Cluster scheduler errors

nwales
Path Finder

Intermittently we're seeing messages similar to the below appear. This is a new search head cluster running 6.2.1 pointing at existing indexers which are running 6.1.2

aid=scheduler_nobodysos_RMD59d4672721e98f163_at_1422416700_1042_E865F266-125C-465F-BFF7-10773D2D3536 on peer=EC6D891C-FF0D-47E9-9D83-864D13A58B04 failed. Leaving it in PendingDiscard state.

Everything else appears to be working ok, so not sure what the issue is here.

jnicholsenernoc
Path Finder

Running 6.2.1 against 6.1.2 is not supported.

http://docs.splunk.com/Documentation/Splunk/6.2.1/DistSearch/SHCsystemrequirements

"All members must run on the same version of Splunk Enterprise. "

0 Karma

markucsb
Explorer

All CLUSTER members need to be on the same version, there is another section that states that 6.2 Search Head Cluster members are backwards compatible with 6.1 Search peers(indexers).

0 Karma

markucsb
Explorer

I'm seeing similar problems with that error message. Additionally my scheduled searches are not consistently firing off at their scheduled times and the dashboards are not reliably retrieving scheduled search results for display. Are you or anyone else experiencing this problem.

0 Karma

nwales
Path Finder

I'm not having too many issues with scheduled searches firing off. Unless there is a search head failure (common with 6.2.1's PDF scheduling bug) in which case things can get backed up.

Also, captain changes can lead to multiple runs of some searches.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...

Design, Compete, Win: Submit Your Best Splunk Dashboards for a .conf26 Pass

Hello Splunkers,  We’re excited to kick off a Splunk Dashboard contest! We know that dashboards are a primary ...

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...