Getting Data In

Forwarder to Indexer secure communication using self signed certs?

newportknight
Loves-to-Learn

Hi,

I am trying to get secure comms between a Forwarder and Indexer up and running using self signed certs but depite following the relevant guides (https://docs.splunk.com/Documentation/Splunk/8.2.1/Security/Howtoself-signcertificates) I keep ending up with the same problem.

I'm generating the self signed cert on a deployment server, creating the RootCA cert, servercert and serverprivate key before transferring them to the Indexer and Forwarder. Once on these I'm creating a newserver cert by combining the 3 files.

I've also created the relevant inputs.conf, outputs.conf and server.conf files using the config guide. It does say to use "password = <string>" in both inputs and outputs conf files but this kicks up an error as it is deprecated so I've used "sslPassword" instead.

After restarting splunkd in the splunkd log on the Indexer I'm getting:

ERROR TcpInputProc - Error encountered for connection from src=10.1.1.34:50772. error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol

I've tried searching for the error and trying various other fixes e.g specifying sslVersions or cipherSuite but I'm still getting the above error.

Could any one offer some help as to where I may be going wrong please?

I've copied the conf files and some outputs from the splund.logs. 

 

Forwarder outputs.conf

[tcpout:group1]

server = 10.1.1.20:9997

disabled = 0

clientCert = /opt/splunk/etc/auth/mycerts/myNewServerCertificate.pem
sslPassword = <key used to generate myServerPrivateKey.key>

useClientSSLCompression = true

 

Forwarder server.conf

[sslConfig]
sslRootCAPath = /opt/splunk/etc/auth/mycerts/myCACertificate.pem

 

Forwarder splunkd log

cat /opt/splunk/var/log/splunk/splunkd.log | grep SSL
07-08-2021 10:19:22.919 +0100 INFO loader - Setting SSL configuration.
07-08-2021 10:19:22.919 +0100 INFO loader - Server supporting SSL versions SSL3,TLS1.0,TLS1.1,TLS1.2
07-08-2021 10:19:46.393 +0100 INFO MongodRunner - Using mongod command line --sslMode requireSSL
07-08-2021 10:19:47.957 +0100 INFO TcpInputProc - Creating fwd data Acceptor for IPv4 port 9997 with Non-SSL

cat /opt/splunk/var/log/splunk/splunkd.log | grep TcpOut
07-08-2021 10:43:42.172 +0100 INFO TcpOutputProc - Found currently active indexer. Connected to idx=10.1.1.20:9997, reuse=1.

 

Indexer inputs.conf

[splunktcp-ssl:9997]
disabled = 0

[SSL]
serverCert = /opt/splunk/etc/auth/mycerts/myNewServerCertificate.pem

sslPassword = <key used to generate myServerPrivateKey.key>

requireClientCert = false

useSSLCompression = false

 

Indexer server.conf

[sslConfig]
sslPassword = $7$YNwWFOGvWECUWkppnTLseT5sGq3wJs72wGEjlZuHDphTK3Jty2nhPQ==

sslRootCAPath = /opt/splunk/etc/auth/mycerts/myCACertificate.pem

 

Indexer splunkd.log

cat /opt/splunk/var/log/splunk/splunkd.log | grep SSL
07-08-2021 10:29:02.382 +0100 INFO ServerConfig - SSL session cache path enabled 0 session timeout on SSL server 300.000
07-08-2021 10:29:02.520 +0100 INFO loader - Setting SSL configuration.
07-08-2021 10:29:02.520 +0100 INFO loader - Server supporting SSL versions SSL3,TLS1.0,TLS1.1,TLS1.2
07-08-2021 10:29:03.093 +0100 INFO MongodRunner - Using mongod command line --sslMode requireSSL
07-08-2021 10:29:04.886 +0100 INFO TcpInputConfig - Creating FwdDataSSLConfig SSL context. Will open port=IPv4 port 9997 with compression=1
07-08-2021 10:29:04.914 +0100 INFO TcpInputConfig - IPv4 port 9997 is reserved for splunk 2 splunk (SSL)
07-08-2021 10:29:04.915 +0100 INFO TcpInputProc - Creating fwd data Acceptor for IPv4 port 9997 with SSL
07-08-2021 10:32:14.117 +0100 ERROR TcpInputProc - Error encountered for connection from src=10.1.1.34:50770. error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
07-08-2021 10:32:14.118 +0100 ERROR TcpInputProc - Error encountered for connection from src=10.1.1.34:50772. error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol


cat /opt/splunk/var/log/splunk/splunkd.log | grep Tcp
07-08-2021 10:29:04.885 +0100 INFO TcpInputConfig - IPv4 port 9997 is reserved for splunk 2 splunk
07-08-2021 10:29:04.886 +0100 INFO TcpInputConfig - Creating FwdDataSSLConfig SSL context. Will open port=IPv4 port 9997 with compression=1
07-08-2021 10:29:04.914 +0100 INFO TcpInputConfig - IPv4 port 9997 is reserved for splunk 2 splunk (SSL)
07-08-2021 10:29:04.915 +0100 INFO TcpInputProc - Creating fwd data Acceptor for IPv4 port 9997 with SSL
07-08-2021 10:29:05.308 +0100 INFO TcpOutputProc - _isHttpOutConfigured=NOT_CONFIGURED
07-08-2021 10:32:14.117 +0100 ERROR TcpInputProc - Error encountered for connection from src=10.1.1.34:50770. error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
07-08-2021 10:32:14.118 +0100 ERROR TcpInputProc - Error encountered for connection from src=10.1.1.34:50772. error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol

Labels (2)
0 Karma

triptraptresko
Explorer

TLDR: I was missing
1. outputs.conf on the sender (client)
useSSL = true

Thank you to michael_bates_1 , in thread https://community.splunk.com/t5/Getting-Data-In/Why-am-I-having-trouble-with-TLS/m-p/634513/highligh...

After following the documentation on how to enable ssl between forwarders and indexers, i got the error

 

ERROR TcpInputProc - Error encountered for connection from src=10.1.1.34:50772. error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol

 

In the documentation: https://docs.splunk.com/Documentation/Splunk/9.0.1/Security/ConfigureSplunkforwardingtousesignedcert...
, it specifies

[SSL]
requireClientCert = true

Which if you drop, will affect your outputs.conf -> useSSL.
It says if requireClientCert is defined, then useSSL will be true.
In my case, I mindlessly thought you could set requrieClientCert=false...

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf
#----Secure Sockets Layer (SSL) Settings---- # To set up SSL on the forwarder, set the following setting/value pairs. # If you want to use SSL for authentication, add a stanza for each receiver # that must be certified. useSSL = <true|false|legacy> * Whether or not the forwarder uses SSL to connect to the receiver, or relies on the 'clientCert' setting to be active for SSL connections. * You do not need to set 'clientCert' if 'requireClientCert' is set to "false" on the receiver. * A value of "true" means the forwarder uses SSL to connect to the receiver. * A value of "false" means the forwarder does not use SSL to connect to the receiver. * The special value "legacy" means the forwarder uses the 'clientCert' property to determine whether or not to use SSL to connect. * Default: legacy

 

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...