topic Re: REST API errors with AMQP Messaging Modular Input add-on in All Apps and Add-ons

REST API errors with AMQP Messaging Modular Input add-on

arthurbreuer — Fri, 30 Jan 2015 16:40:30 GMT

I am using the AMPQ add-on on a RabbitMQ queue. Splunk version is 6.1.

When the add-on starts the next two errors appear in the splunkd.log:

01-29-2015 18:16:51.995 +0100 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/amqp_ta/bin/amqp.py" Can't connect to Splunk REST API with the token [Splunk XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX], either the token is invalid or SplunkD has exited : No appropriate protocol (protocol is disabled or cipher suites are inappropriate)

01-29-2015 18:17:01.997 +0100 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/amqp_ta/bin/amqp.py" It has been determined via the REST API that all inputs have been disabled

The data from the queue is added to the index in Splunk. As far as I can see all REST API functions are working fine.

Do you have any idea what the problem is?

Re: REST API errors with AMQP Messaging Modular Input add-on

Damien_Dallimor — Sat, 31 Jan 2015 15:31:09 GMT

Those are valid messages. The AMQP Modular Input process self manages it's own lifecycle ie: regularly checks if SplunkD has exited , and if so, kills itself.

Re: REST API errors with AMQP Messaging Modular Input add-on

arthurbreuer — Sat, 31 Jan 2015 23:43:35 GMT

Hi Damien,

Thank you for your reply. The problem is that splunkd is still active. When I only restart the add-on the one item from the queue is added to Splunk and the same errors appear again.

Regards,

Arthur

Re: REST API errors with AMQP Messaging Modular Input add-on

Damien_Dallimor — Sun, 01 Feb 2015 05:08:50 GMT

Does your OS correctly resolve "localhost" to the IP that SplunkD is bound to ?
Also , what version of Splunk are you on and what Java Runtime version are you using ?

Re: REST API errors with AMQP Messaging Modular Input add-on

arthurbreuer — Mon, 28 Sep 2020 18:49:07 GMT

Hi Damien,

The server is part of a domain. Localhost is resolved correctly. Ping, traceroute, wget, everything works.

When I link localhost to a non existing IP, the following error appears:
02-02-2015 15:21:15.716 +0100 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/amqp_ta/bin/amqp.py" Probing socket connection to SplunkD failed.Either SplunkD has exited ,or if not, check that your DNS configuration is resolving your system's hostname (xxxx.xxxx) correctly : Connection refused

And later, the two errors appear again:
02-02-2015 15:21:17.313 +0100 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/amqp_ta/bin/amqp.py" Can't connect to Splunk REST API with the token [Splunk XXXXXXXXXXXXXXXX], either the token is invalid or SplunkD has exited : Connection refused
02-02-2015 15:21:27.314 +0100 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/amqp_ta/bin/amqp.py" It has been determined via the REST API that all inputs have been disabled

Our server virtualized and is running on Splunk version 6.1. I also tested it on Splunk version 6.2.1 on a testserver. As for Java, I tried it on Java openjdk 1.7.0_75 and Java SE 1.8.0_31 (Oracle).

Re: REST API errors with AMQP Messaging Modular Input add-on

arthurbreuer — Thu, 05 Feb 2015 11:06:37 GMT

Any ideas Damien?

Re: REST API errors with AMQP Messaging Modular Input add-on

Damien_Dallimor — Thu, 05 Feb 2015 11:58:35 GMT

The Splunk Java SDK code that is being used to perform the Splunk callbacks that are leading to the error message you posted is attempted to use SSLv3. Can you enable this in your setup ?

http://docs.splunk.com/Documentation/Splunk/6.2.1/Admin/Serverconf (look at the sslConfig stanza)

Re: REST API errors with AMQP Messaging Modular Input add-on

arthurbreuer — Mon, 28 Sep 2020 18:50:48 GMT

It looks like SSLv3 is enabled. But still the errors appear.

server.conf

[sslConfig]
enableSplunkdSSL = true
useClientSSLCompression = true
useSplunkdClientSSLCompression = true

supportSSLV3Only = true
sendStrictTransportSecurityHeader = true
allowSslCompression = true
allowSslRenegotiation = true

cipherSuite = ALL:!aNULL:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM

sslKeysfile = server.pem
sslKeysfilePassword = XXXXXXXXXXXXXXXXXXXXXX
caCertFile = cacert.pem
caPath = $SPLUNK_HOME/etc/auth
certCreateScript = $SPLUNK_HOME/bin/splunk, createssl, server-cert

web.conf

[default]

[settings]

startwebserver = 1
httpport = 443
enableSplunkWebSSL = true
supportSSLV3Only = true

Re: REST API errors with AMQP Messaging Modular Input add-on

arthurbreuer — Fri, 06 Feb 2015 17:27:00 GMT

Hi Damien,

Good news this time! The problem is that SSLv3 is disabled in Java by default. The same problem is mentioned in http://answers.splunk.com/answers/209379/no-appropriate-protocol-protocol-is-disabled-or-ci.html

The only thing you have to do is to enable SSLv3 (if this is possible off course). Just add a comment ('#') in front of the last line of the file "java.security". You can find this file in the directory:

OpenJDK : /usr/lib/jvm-java--openjdk-.x86_64/jre/lib/security
Oracle SDK : /usr/java/jdk/jre/lib/security

If you agree that this is the solution, maybe you can make a note in your documentation?

And again, thank you for your help.

Re: REST API errors with AMQP Messaging Modular Input add-on

Damien_Dallimor — Sun, 08 Feb 2015 17:22:30 GMT

The latest release has TLSv1.2 support wired in. View the release notes for the latest release for how to enable TLS.

Re: REST API errors with AMQP Messaging Modular Input add-on

pkhalsa — Mon, 09 Feb 2015 18:30:07 GMT

Hi Damien, I'm kind of new to Splunk and when you write in the release notes:

"To do so you specify "splunk.securetransport.protocol=tls" in the Additional JVM System Properties parameter when you configure the stanza."

I'm not sure to which stanza you are referring, or where to find it. Could you give some more guidance?
Thanks.

Re: REST API errors with AMQP Messaging Modular Input add-on

Damien_Dallimor — Mon, 09 Feb 2015 19:09:29 GMT

Disregard , I rolled back the newest release , found a bug.

Re: REST API errors with AMQP Messaging Modular Input add-on

pkhalsa — Mon, 09 Feb 2015 19:17:43 GMT

Wow, thanks for the quick response. I'm getting the same error as Arthur, but for some reason, I'm not able to edit java.security in a way that allows SSLv3, i.e. I keep getting the same error.

Re: REST API errors with AMQP Messaging Modular Input add-on

Damien_Dallimor — Mon, 09 Feb 2015 19:23:26 GMT

unless you can enable SSLv3 , I can't do much right now.

Re: REST API errors with AMQP Messaging Modular Input add-on

pkhalsa — Mon, 09 Feb 2015 20:02:13 GMT

Arthur, after reading this thread last week, I realized that I needed to comment out the option in java.security. However, when I went to that file, I realized it was already commented out. Yet I still get the error that SSLv3 is disabled by Java and I can't figure out where else it could possibly disabled. I thought perhaps it was somehow disabled within one of the jar files of the app, but I guess Damien would have mentioned that. Do you know of any other place where Java might disable SSLv3?
Thanks for any help.

Re: REST API errors with AMQP Messaging Modular Input add-on

arthurbreuer — Mon, 09 Feb 2015 20:50:14 GMT

Hi Pkhalsa,

You are right. The last line has to be commented out, so it should look like this: #jdk.tls.disabledAlgorithms=SSLv3.

Is it possible that you have more that one java installed?

Re: REST API errors with AMQP Messaging Modular Input add-on

Damien_Dallimor — Tue, 10 Feb 2015 09:07:04 GMT

Update : try the version here : http://damiendallimore.github.io/ (before I release it and screw up Splunkbase again 🙂 )

I weeded out the TLS bug / SDK incompatibilitys issues

Re: REST API errors with AMQP Messaging Modular Input add-on

pkhalsa — Tue, 10 Feb 2015 20:36:18 GMT

That line looked different in my java.security file and it was already commented out. I resolved this issue in a kind of brute force manner, in that I installed the latest java and commented out the line in the latest java. Now I'm getting a different error 😛

Re: REST API errors with AMQP Messaging Modular Input add-on

pkhalsa — Tue, 10 Feb 2015 20:48:22 GMT

I managed to enable SSLv3 by updating Java and doing the same thing I did before, but this time it worked.
I hope you don't mind a follow on question. Since I'm still a newbie, I wanted to ask you about the following error, which I'm now receiving:
"Can't connect to Splunk REST API with the token..."
SplunkD hasn't exited, so I'm wondering why it would get an invalid token?
Thanks.