Splunk monitoring console unable to gather Resource usage information

[PeraltaRH]

Hello,

I have a problem with my distributed environment where some of my instances appear greyed out under CPU and memory utilization. If I open the specific instance panels I see the red icon with the errors below. 

This is very strange because it affects the indexers only. The only difference with them is that the web service is disabled a per Splunk best practices.

I tried with splunk support by opening a case but we couldn't find the solution yet.

Splunk Version: 8.0.3 (indexers), 8.0.5 (everything else)--- Is this a problem? 

[subsearch][servername]Failed to fetch REST endpoint uri=https://127.0.0.1:8089/services/server/info?count=0&strict=false from server https://127.0.0.1:8089. Check that the URI path provided exists in the REST API.

[subsearch][servername]Unexpected status for to fetch REST endpoint uri=https://127.0.0.1:8089/services/server/info?count=0&strict=false from server=https://127.0.0.1:8089 - Bad Request

--- MORE INFO ---

If I run the command manually like:

https://10.10.10.1:8089/services/server/status/resource-usage/hostwide
I get the output in my browser.

I read this post: https://community.splunk.com/t5/Getting-Data-In/Splunk-Management-Console-Error-subsearch-Rest-Proce... 

It talks about the indexer role, my Cluster Master is also "SHC Deployer" (Search Head Cluster Deployer), would this be the role I have to move? Its not an Indexer, I have 6 dedicated indexers and 5 dedicated search heads.

[PeraltaRH · Accepted solution]

The problem seems to be with the versions compatibility.

A bit absurd that a new MC version is not retro-compatible, not even a mayor upgrade 8.0.3 vs 8.0.5.1.

If anyone gets this problem, its because the new version sends "strict=false" and the old version does not like it. You cant even set the MC to not send this option.... quite disappointing. 

[PeraltaRH]

For reference: 

https://community.splunk.com/t5/All-Apps-and-Add-ons/8-0-6-REST-API-call-fails-with-error-after-upgr...