Questions about UF outputs.conf Configuration

[tt-nexteng]

I am configuring TLS communication between UF (Universal Forwarder) and Indexer.
My outputs.conf configuration is as follows:

[tcpout]
defaultGroup = default-autolb-group

[tcpout-server://xxxxxxx:9997]

[tcpout:default-autolb-group]
server = xxxxxxx:9997
disabled = false
sslPassword = ServerCertPassword
sslRootCAPath = /opt/splunkforwarder/etc/auth/mycerts/myCertAuthCertificate.pem
sslVerifyServerCert = false
useACK = true
sslCertPath = /opt/splunkforwarder/etc/auth/mycerts/myCombinedServerCertificate.pem

I have three questions:

1. I don't need a client certificate right now. If I don't set sslCertPath, an error occurs. Is this option mandatory?
2. Currently, I have set sslCertPath to the server certificate, and TLS communication works. Why do I need to set the server certificate on the client? Is this a common practice?
3. If I want to use a client certificate, which configuration setting should I use?

[livehybrid]

Hi @tt-nexteng 

Do you have requireClientCert set within your inputs.conf file on your receiving Splunk instance?

sslCertPath in the outputs.conf is actually deprecated and clientCert should be specified instead, although obviously this is only if you intend to use MutualAuth.

sslRootCAPath in the outputs.conf is also deprecated and instead should be set in server.conf under the [sslConfig] stanza. Perhaps the CA isnt being picked up by the output processor and therefore it is using the combined cert you have specified in the sslCertPath.

Try updating your server.conf/[sslConfig]/sslRootCAPath to your CA file and then try to see if this resolves the issue.

Please let me know how you get on and consider upvoting/karma this answer if it has helped.
Regards

Will

[tt-nexteng]

This is 2017 post

https://community.splunk.com/t5/Getting-Data-In/inputs-conf-and-outputs-conf-for-SSL-encryption/m-p/...

[tt-nexteng]

When I asked this question, I had already added the following setting under [sslConfig] in both my Indexer and UF's server.conf:

sslRootCAPath = /opt/splunkforwarder/etc/auth/mycerts/myCertAuthCertificate.pem

However, I still encountered the same issue as described in my original question.

Additionally, my Indexer's inputs.conf is configured as follows:

[splunktcp-ssl:9997]
disabled = 0

[SSL]
serverCert = /opt/splunk/etc/auth/mycerts/myCombinedServerCertificate.pem
sslPassword = ServerCertPassword
requireClientCert = false

I have followed Splunk's official documentation and tried various configurations, but all attempts failed.

Then, I found a 2017 post on the Splunk Community forum and decided to try the suggested configuration.
That configuration is exactly what I am using now, and it worked successfully.

I don't fully understand this configuration, so I have asked these three questions.

[PickleRick]

Ok. You have two ends of the connection, don't try to fiddle with both of them at the same time.

First, configure the receiving end (in your case - the indexer), when you have it working properly, start configuring the client (the UF).

Your inputs.conf on the indexer looks OK.

You should now be able to connect with

openssl s_client -connect your_indexer:9997

and get a properly negotiated SSL connection (as long as your client trusts your indexer's cert issuer).

If you're at this step, you can move forward. If at this step the connection is rejected by the indexer because you're not presenting a cert, there's something wrong with your indexer's configuration.

If you have sslVerifyServerCert=false, you should not need any other parameters except useSSL=true because your UF will not be verifying the cert anyway.

Remember to always check your configs with btool

splunk btool check 

and

splunk btool inputs list --debug
splunk btool outputs list --debug

[tt-nexteng]

Thank you very much. I followed your suggestion and modified output.cnf as follows, and it worked successfully.

[tcpout]
defaultGroup = default-autolb-group

[tcpout-server://52.195.142.152:9997]

[tcpout:default-autolb-group]
server = 52.195.142.152:9997
disabled = false
sslVerifyServerCert = false
useSSL = true

I would like to ask, if I want to use a client certificate, and my server certificate's CN is splunk.xx.net, while my client certificate's CN is uf.xx.net, how should I configure the output and input settings?

Additionally, I want both the server and client to mutually verify each other's certificates.Could you give me a  sample.

[PickleRick · Accepted solution]

If you're not explicitly limiting allowed clients to a predefined list, CNs and SANs in the certs don't matter (as long as the certs are not self-signed which means that CN of the CA is the same as CN of the issued cert).

If you do verify server name (sslVerifyServerName setting) there are additional restrictions that the name in the cert presented by the host must match the hostname you're trying to connect to. But at this point you're not using this.

So the first thing to enable is to verify server's cert. For this you need to have CA defined on your UF (preferably by setting sslRootCAPath in your server.conf) containing a PEM-encoded certificate of the CA which issued either the indexer's cert directly or is the rootCA from which the indexer's cert is descended.

Then you enable sslVerifyServerCert.

If at this point UF cannot connect to the indexer, there's something wrong with the trust relationship between indexer and UF. Check logs. Sometimes it helps to do a tcpdump and see where exactly the connection gets terminated and with what alert.

If you manage to get server verification working, time to enable client authentication.

You have

sslRootCAPath = /opt/splunkforwarder/etc/auth/mycerts/myCertAuthCertificate.pem

in your inputs.conf (actually this setting is deprecated and you should use the setting from server.conf; if you don't have a separate different setting there, we might leave it at this moment; If you do - I have no idea how Splunk reacts).

That means that you need the client (UF) to present a valid certificate on connection attempt.

clentCert = /path/to/your/crypto_material.pem

Should be enough on the UF end as long as the key is not encrypted. If it is, you need to set sslPassword.

The PEM file must be in the form of client certificate, client private key, certification chain (optionally) all concatenated into a single file.

Then on the indexer's end you simply enable requireClientCert.

And you're good to go.

Again - don't do too many things at once. One step at a time. And remember to have valid certificates (properly issued, not self-signed, not expired and so on).

[tt-nexteng]

Thank you very much.
I followed your method and resolved the issue.