A few weeks ago the OS was updated, and the Splunk HF was updated to 6.3. This was Splunk VMWare 3.1.0, but have updated to 3.2.1 to no affect.
I noticed today that neither the performance, nor inventory data is being collected. In the logs I see messages about not being able to connect:
2016-03-11 19:07:11,366 ERROR [ta_vmware_collection_worker://theta:10258] Problem with hydra worker ta_vmware_collection_worker://theta:10258: [HydraGatewayAdapter] could not authenticate with gateway after 3 retries
Traceback (most recent call last):
File "/app/splunk/etc/apps/SA-Hydra/bin/hydra/hydra_worker.py", line 507, in run
self.establishGateway()
File "/app/splunk/etc/apps/SA-Hydra/bin/hydra/hydra_worker.py", line 428, in establishGateway
hga = HydraGatewayAdapter(self.node_path, self.session_key, self.gateway_uri)
File "/app/splunk/etc/apps/SA-Hydra/bin/hydra/hydra_common.py", line 66, in init
self.authenticate_gateway()
File "/app/splunk/etc/apps/SA-Hydra/bin/hydra/hydra_common.py", line 83, in authenticate_gateway
raise Exception("[HydraGatewayAdapter] could not authenticate with gateway after %s retries" % (str(retry)))
Exception: [HydraGatewayAdapter] could not authenticate with gateway after 3 retries
Other indications that it's should be connecting on port 8008, but there's no listening process. troubleshooting ideas, or resolution?
... View more