Splunk Search

Splunk using cipherSuite caused CLI command issues


I am on Splunk Version : 6.1.3 and trying to use splunk supported cipherSuite from TLSv1.2, but it is causing the CLI command to fail and as a result he is unable to posh the cluster bundle or use the REST end point of the Peer.

Here is Configuration:

../etc/system/local/server.conf [sslConfig]
../etc/system/local/server.conf allowSslCompression = false
../etc/system/default/server.conf allowSslRenegotiation = true
../etc/system/default/server.conf caCertFile = cacert.pem
../etc/system/default/server.conf caPath = ../etc/auth
../etc/system/default/server.conf certCreateScript = ../bin/splunk, createssl, server-cert
../etc/system/local/server.conf cipherSuite = AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-DSS-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDH-RSA-AES256-GCM-SHA384:ECDH-ECDSA-AES256-GCM-SHA384
../etc/system/local/server.conf enableSplunkdSSL = true
../etc/system/default/server.conf sendStrictTransportSecurityHeader = false
../etc/system/default/server.conf sslKeysfile = server.pem
../etc/system/local/server.conf sslKeysfilePassword = #####
../etc/system/local/server.conf supportSSLV3Only = True
../etc/system/local/server.conf useClientSSLCompression = false
../etc/system/local/server.conf useSplunkdClientSSLCompression = false

Issue is that CLI command or REST endpoints is failing with error :

./splunk list monitor result in error *

Splunk is not running, and it must be for this operation. To start splunk, run "splunk start". (02)

CLI command to check the bundle status also result in error
splunk show cluster-bundle-status

*Failed to contact the master. ERROR:
Couldn't complete HTTP request: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshak*e failure

The REST endpoint gives error
curl -u admin:changeme -k https://localhost:8089/services/search/jobs -d"search=search *"

curl: (35) SSL connect error

0 Karma

New Member

I got this error whilst trying to disable the obsolete (according to Google Chrome) RSA key exchange ciphers by removing the "RSA+AESGCM:RSA+AES" from my cipherSuite.

Upon investigating this issue, it seemed to be caused by the fact that out of the box, splunk cannot use the ECDH ciphers (ecdhCurves = ), so it always does a fallback to the ciphers with RSA as key exchange. But if you remove the RSA ciphers, there are no ciphers left for splunk to use, so it fails.

By specifying "ecdhCurves = prime256v1,secp384r1,secp521r1" the ECDH ciphers can be used, and solves the problem.

Using this in combination with the following config enables HSTS headers and Perfect Forward Secrecy:

ecdhCurves = prime256v1,secp384r1,secp521r1
sendStictTransportSecurityHeader = true
sslVersions = tls1.2

The above configuration is applicable to server.conf under [sslConfig] for the mgmt and kv store port, and in web.conf under [settings] for splunkweb.
(The ciperSuite is taken form the Mozilla SSL Configuration Generator using the modern profile. Older browsers may not support this)

0 Karma

Splunk Employee
Splunk Employee

I've been able to recreate it on my side using the following:

  1. Clean install of Splunk 6.2
  2. Update server.conf with the following:


  3. Restarted Splunk

  4. Ran the following command:

    [root@bporepo01 local]# /opt/splunk/bin/splunk list monitor
    Couldn't complete HTTP request: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Re-opening the bug.

New Member

So how do we do this LD Library Setting?


0 Karma


For us it was "export LD_LIBRARY_PATH=/opt/splunk/splunkforwarder/lib"

But it is based off of your install path for the splunk lib directory it would most likely be:
export LD_LIBRARY_PATH=/opt/splunkforwarder/lib

srry for the delay in response

0 Karma

Splunk Employee
Splunk Employee

For new install of Splunk 6.2 you need to add following to web.conf

supportSSLV3Only = false

Path Finder

Hope people in need will see this. Had the same issue while trying to enable Search Head Clustering and had to add the above configs to my web.conf as well as server.conf under [sslConfig]


we also needed the settings in both web and server.conf. in addition to the LD_LIBRARY setting.

Splunk Employee
Splunk Employee

My issue was a different issue - with HP-UX install, LD_LIBRARY has to be set before attempting any CLI commands.

Rajpal again rocks the house.

0 Karma


Yes that also resolved the issue for us, Rajpal does indeed rock!

Splunk Employee
Splunk Employee

This issue has been fixed in Splunk Version 6.2. Please consider upgrade.


um...we just upgraded to splunk 6.2 and we are having the same issue as outlined above. @rbal_splunk are you SURE this was fixed in 6.2 release because it does not seem to be:

$ splunk list forward-server
Couldn't complete HTTP request: Could not find SSL library

$ splunk show cluster-bundle-status

Failed to contact the master. ERROR:
Couldn't complete HTTP request: Could not find SSL library

Failed to contact the peers endpoint. ERROR:
Couldn't complete HTTP request: Could not find SSL library

0 Karma


Hi @mookiie2005

There was some confusion with the postings on this question so I just cleaned everything up. Please refer to @bosburn_splunk's answer and @rbal_splunk's comment below that

0 Karma
Get Updates on the Splunk Community!

Dashboard Studio Challenge - Learn New Tricks, Showcase Your Skills, and Win Prizes!

Reimagine what you can do with your dashboards. Dashboard Studio is Splunk’s newest dashboard builder to ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...