Indexer forwarder ssl communication CRL not used

[Roman_Micek]

Hi guys.

I have a problem with certificate revocation on Splunk forwarder.

Description:

There are 3 VM with Red Hat:

1. Certification Authority (CA) -  with Easy RSA installed and  Apache server to publish certificates
2. Indexer (IDX)- full Splunk server installation
3. Forwarder(FW) - Splunk forwarder  

I managed to create certificate for both IDX and FW then signed them using EasyRSA on CA. System is able to establish SSL connection  between IDX and FW. So far I am HAPPY. But when I use CA to revoke FW certificate Splunk is  not able to detect this change and  system still takes FW certificate as valid. 

After reworking FW certificate I have published the new CRL in /var/www/pki/crl.pem . Using browser i am able  to download it and check that certificate was revoked.  From /var/log/httpd/access_log I can tell that IDX or FW have never accessed the CRL.

I tried to set sslCommonNameToCheck. This works fine but it is unsuitable  for me because the final solution has hundreds of Forwarders and maintaining the list in sslCommonNameToCheck is too cumbersome.

Also tried splunk reload crl with no success.

File Settings:

IDX(server.conf)

[sslConfig]
sslRootCAPath = /opt/splunk/etc/auth/myauth/ca.pem

IDX(inputs.conf)

[splunktcp-ssl:9997]
disabled = 0

[SSL]
serverCert = /opt/splunk/etc/auth/myauth/myNewServerCertificate.pem
sslPassword = $7$qV7bjcVNcqRlm70Y1cpaazqeGFmH6nyfnNN1TSCDu82ZPhnqMw==
requireClientCert = true

FW(server.conf)

[sslConfig]
sslRootCAPath = /opt/splunkforwarder/etc/auth/myauth/ca.pem

FW(outputs.conf)

[tcpout]
defaultGroup = indexer2

[tcpout:indexer2]
server = xx.xx.xx.xx:9997
clientCert = /opt/splunkforwarder/etc/auth/myauth/myNewClientCertificate.pem
sslPassword = $7$hibYhkL2wOexhWDmyBqMEk358HGFaLe4jQ8RT6ruDsEeQmS6Ww==

Thank you for your time and so much needed advice.

[Roman_Micek]

Thank you.

It was not the answer I was hoping for  but at least I know whats'up.

[tscroggins · Accepted solution]

@Roman_Micek 

A great reference for properly configuring TLS in Splunk is the Splunk Enterprise 8.1 Common Criteria Configuration Guide at https://www.niap-ccevs.org/MMO/Product/st_vid11108-agd.pdf. The guide changes the SPLUNK_ETC location, but you can replace those references with $SPLUNK_HOME/etc as needed.

CRLs should be downloaded using a process you define to $SPLUNK_HOME/etc/auth/crl. Splunk does not automatically download CRLs, and Splunk does not support OCSP.

KVStore CRL checking is configured separately in server.conf.

Note that CRLs are only checked if you installed and configured Splunk with SPLUNK_FIPS=1 in splunk-launch.conf. (You don't need SPLUNK_COMMON_CRITERIA=1 unless that's your goal.) If you did not enable FIPS mode before running Splunk for the first time, you'll need to reinstall Splunk. This applies to both Splunk Enterprise and Splunk Universal Forwarder.