How to get tcp-ssl input for Splunk 6.0 to work

[ricktao]

I have installed Splunk 6.0 (Free version) on Linux x64 system.
I can collect syslog inputs on UDP port 514. But I tried to add "tcp-ssl", it didn't work.
Here are my contents in /opt/splunk/etc/apps/launcher/local/inputs.conf file:

[udp://514]
connection_host = ip
source = SyslogTest
sourcetype = syslog

[tcp-ssl:10514]
source = SyslogSslTest
sourcetype = syslog
disabled = 0

[SSL]
serverCert = /root/splunk/cert.pem
rootCA = /root/splunk/ca.pem
password = $1$jC3aVtsP5w==

I did "/opt/splunk/bin/splunk btool check --debug", I didn't see anything wrong.
I did "/opt/splunk/bin/splunk start" to run "OK". Even the web interface shown tcp inputs.
But if I did "netstat -a | grep 514", I didn't see any my SSL port 10514 open.

Please advice any tips to run "syslog over ssl/tls" or debug the problem. Thanks!

[TonyLeeVT]

Here is how to do this in Windows:

NOTE: sslRootCAPath is ignored in Windows. Instead use: caCertFile (Thank you Splunk support....)

Create the certs:

mkdir c:\progra~1\Splunk\etc\certs
C:\progra~1\Splunk\bin\splunk.exe cmd cmd.exe /c c:\progra~1\Splunk\bin\genRootCA.bat -d c:\progra~1\Splunk\etc\certs
C:\progra~1\Splunk\bin\splunk.exe cmd python c:\progra~1\Splunk\bin\genSignedServerCert.py -d c:\progra~1\Splunk\etc\certs -n splunk -c splunk -p

Add the following to: c:\Program Files\Splunk\etc\system\local\server.conf

[sslConfig]
caCertFile = c:\progra~1\Splunk\etc\certs\cacert.pem

Add the following to: c:\Program Files\Splunk\etc\apps\your_app_here\local\inputs.conf

[tcp-ssl://6514]
disabled = false
sourcetype = <optional>
index = <optional>
source = <optional>

[SSL]
sslPassword = <The password that was used in the genSignedServerCert>
requireClientCert = false
serverCert = c:\progra~1\Splunk\etc\certs\splunk.pem

Restart Splunk:
c:\progra~1\Splunk\bin\splunk.exe restart

Now verify the port is open using:

netstat -an | findstr :6514

[aornitops]

This worked flawlessly for me in 7.3.1! Thanks for posting. I did see this while going through the process:

C:\Windows\system32>C:\progra~1\Splunk\bin\splunk.exe cmd python c:\progra~1\Splunk\bin\genSignedServerCert.py -d c:\progra~1\Splunk\etc\certs -n splunk -c splunk -p
**NOTE: This script is deprecated.  Instead, use "splunk createssl server-cert"**

Does anyone know if the commands are the same for "splunk createssl server-cert"?

[TonyLeeVT]

After checking splunkd.log and a little troubleshooting, I was able to get the tcp-ssl port to listen and receive ssl encrypted traffic from a third party device. I used the following steps:

Generate certs:

mkdir /opt/splunk/etc/certs
export OPENSSL_CONF=/opt/splunk/openssl/openssl.cnf
/opt/splunk/bin/genRootCA.sh -d /opt/splunk/etc/certs

/opt/splunk/bin/genSignedServerCert.sh -d /opt/splunk/etc/certs -n splunk -c splunk -p

**Note: It will ask you to enter a password

In inputs.conf, I used the following:

[tcp-ssl://6514]
Sourcetype = <your source type here>

[SSL]
rootCA = $SPLUNK_HOME/etc/certs/cacert.pem
serverCERT = $SPLUNK_HOME/etc/certs/splunk.pem
password = <The password that was used in the genSignedServerCert>

You may want to use netstat -an | grep :6514 to make sure the port is listening after a restart. If not, check /opt/splunk/var/log/splunkd/splunkd.log again for errors. Hope that helps!

[TonyLeeVT]

Wow... what should be a simple feat is really a moving target. It appears the process has been changed again in Splunk 6.5.0+

Here is what you need to do now:

Generate certs:

 mkdir /opt/splunk/etc/certs
 export OPENSSL_CONF=/opt/splunk/openssl/openssl.cnf
 /opt/splunk/bin/genRootCA.sh -d /opt/splunk/etc/certs

 /opt/splunk/bin/genSignedServerCert.sh -d /opt/splunk/etc/certs -n splunk -c splunk -p

$SPLUNK_HOME/etc/apps//local/inputs.conf:

[tcp-ssl://6514]
disabled = false
sourcetype = <optional>
index = <optional>
source = <optional>

[SSL]
serverCert = /opt/splunk/etc/certs/splunk.pem
sslPassword = <The password that was used in the genSignedServerCert>
requireClientCert = false

$SPLUNK_HOME/etc/system/local/server.conf:

[sslConfig]
sslRootCAPath = /opt/splunk/etc/certs/cacert.pem

Restart Splunk:

$SPLUNK_HOME/bin/splunk restart splunkd

Verify the port is open:

netstat -an | grep :6514

[TonyLeeVT]

Note: Splunk seems to have deprecated the password parameter. It is now: sslPassword.

Also note: You may need to replace $SPLUNK_HOME with /opt/splunk or whatever your path may be...

Pro-tips:
- Check the port status with netstat -an | grep :6514 (following our above example)
- You may also want to use this command for troubleshooting: /opt/splunk/bin/splunk btool check --debug
- If you are still having issues with the port opening, try to retype the sslPassword and save the file. If you mistype the password, this will not work.

[ricktao]

Now I finally found the answer in the following website:

Use the Splunk provided tools to generate CA, and Server Certificates.

AND you have use $SPLUNK_HOME instead of /opt/splunk/..

http://answers.splunk.com/answers/7164/how-do-i-set-up-ssl-forwarding-with-new-self-signed-certifica...

[Simeon]

Something probably didn't get picked up if the netstat didn't produce anything. I would try enabling the same stanza with standard TCP to ensure functionality, then convert it to SSL. Here is a thread on the topic also in answers:

http://answers.splunk.com/answers/51707/how-to-configure-my-splunk-app-to-get-data-over-ssl

http://answers.splunk.com/answers/48955/tlsssl-syslog-splunk-support

[ricktao]

I followed the following URL to create the cert, not sure where it get wrong:

http://docs.splunk.com/Documentation/Splunk/5.0/Security/Howtoself-signcertificates

[ricktao]

SSL still not work, but found ERRORs in /opt/splunk/var/log/splunk/splunkd.log

splunkd.log: ERROR SSLCommon - Can't read CA list
splunkd.log: ERROR TcpInputConfig - SSL server certificate not found, or password is wrong - SSL ports will not be opened
splunkd.log: ERROR TcpInputConfig - SSL context not found. Will not open raw (SSL) IPv4 port 6514

It seems that I used the plaintext like "abc123" for password, and changed to like "$1$jC3aVtsP5w==" after startup
Maybe that's the cause of the TCP-SSL problem.

[SSL]
serverCert = /root/cert.pem
rootCA = /root/ca.pem
password = $1$jC3aVtsP5w==