Splunk Search Head Cluster scheduler errors

[nwales]

Intermittently we're seeing messages similar to the below appear. This is a new search head cluster running 6.2.1 pointing at existing indexers which are running 6.1.2

aid=scheduler_nobodysos_RMD59d4672721e98f163_at_1422416700_1042_E865F266-125C-465F-BFF7-10773D2D3536 on peer=EC6D891C-FF0D-47E9-9D83-864D13A58B04 failed. Leaving it in PendingDiscard state.

Everything else appears to be working ok, so not sure what the issue is here.

[jnicholsenernoc]

Running 6.2.1 against 6.1.2 is not supported.

http://docs.splunk.com/Documentation/Splunk/6.2.1/DistSearch/SHCsystemrequirements

"All members must run on the same version of Splunk Enterprise. "

[markucsb]

All CLUSTER members need to be on the same version, there is another section that states that 6.2 Search Head Cluster members are backwards compatible with 6.1 Search peers(indexers).

[markucsb]

I'm seeing similar problems with that error message. Additionally my scheduled searches are not consistently firing off at their scheduled times and the dashboards are not reliably retrieving scheduled search results for display. Are you or anyone else experiencing this problem.

[nwales]

I'm not having too many issues with scheduled searches firing off. Unless there is a search head failure (common with 6.2.1's PDF scheduling bug) in which case things can get backed up.

Also, captain changes can lead to multiple runs of some searches.