The script jmx.py can't connect to server splunk socket. My splunk server is binding another IP address (configure in etc/splunk-launch.conf) than server hostname. Where can I configure this setting ?
Error in splunkd.log:
09-25-2014 16:18:41.626 +0200 ERROR ExecProcessor - message from "python /@mysplunkhome@/splunk/etc/apps/jmx_ta/bin/jmx.py" Probing socket connection to SplunkD failed.Either SplunkD has exited ,or if not, check that your DNS configuration is resolving your system's hostname (@myhostname@) correctly : Connection refused
The JMX Mod Input does phone homes to SplunkD to check that SplunkD is still running.
It does this by way of a simple socket connection attempt to the SplunkD management TCP port
You'll get the above error message written if SplunkD is genuinely not running or if your SplunkD host's naming is not resolving (perhaps due to your local DNS settings)
Ok, but SplunkD is not listening on the host IP address because I use "SPLUNK_BINDIP" directive in splunk-launch.conf to bind to another IP address.
We hardened our Splunk Universal Forwarder install by having it listen on 127.0.0.1:8089 instead of splunkhostname:8089 and this does break the plugin (monitoring-of-java-virtual-machines-with-jmx_24).
I found this:
Is it possible to add logic to also check for "SPLUNK_BINDIP=x.x.x.x" in /etc/splunk-launch.conf?
The Mod Input gets the splunkhostname from SplunkD when the Mod Input is instantiated.
So perhaps there is something that you can do with your local DNS to resolve splunkhostname -> 127.0.0.1
Since fiew months, I have an ugly workaround, redirect all request on 8089 port to x.x.x.x:8089 :
iptables -t nat -A OUTPUT -p tcp -o lo --dport 8089 -j DNAT --to x.x.x.x:8089