Hey all,
I am in the process of migrating from a Windows Heavy Forwarder to a Linux Heavy Forwarder for Splunk Cloud. Part of this exercise involves migrating the Splunk DB Connect App from the Windows HF Box to the new Red Hat 8.4 HF box.
Quick Details:
I basically duplicated the configuration from the original Windows Server that ran DBConnect App. I brought over the same connection information as well as the same identity information. I've validated that the identity information is correct. I am getting the following error:
Database connection server.domain.com is invalid.
The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption.
Error: "Certificates do not conform to algorithm constraints".
This seems to imply that there is some sort of certificate negotiation error. I have browsed through the DBConnect documentation but nothing inside there seems to help. I noticed a few different keystores around the db connect app and I tried messing with a few of them, but without luck. Currently in the "keystore" folder, I have loaded a domain PKI cut/issued cert/key pair and the domain PKI CA chain. None of those seem to make any difference. My basic connection string looks like the following in the edit url box:
jdbc:sqlserver://server.domain.com:1433;databaseName=Splunk;selectMethod=cursor;encrypt=true
I've tried various differentiations of this as well like:
jdbc:sqlserver://server.domain.com:1433;databaseName=Splunk;selectMethod=cursor;encrypt=true;trustStore=/opt/splunk/etc/apps/splunk_app_db_connect/keystore/default.jks;trustStorePassword=password
I wasn't sure how to configure the JRE installation path and I also wasn't too positive on where it was located in the Red Hat 8.4 instance. I did some tracking and I think it's loaded here:
/usr/lib/jvm/java-11-openjdk-11.0.12.0.7-0.el8_4.x86_64/
I mainly did that because it appears like JAVA_HOME wasn't set in the OS. I could have set it but I figured it would take out any potentially issues if I just pointed it directly at the folder.
I haven't had much luck. I loaded up wireshark and confirmed I could see the connection and I do see the inbound 1433 connection from the heavy forwarder. I do see the active connection being made, but because it's over 1433 Wireshark isn't showing any TLS negotiation. I am not sure if that's an issue or not.
I am not sure where else to go from here. Does anyone have any thoughts?
I've figured out the issue. I had to take ownership over the SQL Server. After I got on it I realized 2 things.
So I attempted to put a certificate on the SQL Server service however, the service wouldn't start. After some further research, the service won't start if its being run as a local account. I created a network account and gave it the proper permissions and then applied it to the SQL Server Service. Once I did that then I started service successfully and the original error message disappeared.
I am now running into a different issue trying to test the connections however, it looks like I am running into a known issue (https://community.splunk.com/t5/All-Apps-and-Add-ons/Splunk-DBConnect-3-1-1-Build-34-There-was-an-er...)? Can someone confirm? Error message:
There was an error processing your request. It has been logged (ID ddc19c6c869a60ee)
Port number has nothing to do with Wireshark showing TLS negotiation or not (at least in the typical configuration unless you did something bad to it 😉). Even if it wasn't decoding the TLS setup itself you should at least be able to spot some of the certificate data (like DN or CA name) within the packets being exchanged.
If you don't it's probable that no TLS negotiation is taking place.
It appears you have to dig pretty deep to see any sort of TLS info. It's not a standard TLS protocol setup by any means. I see the following two packets in succession:
It appears like a TLS connection is being attempted:
Still can't tell if it's being rejected because of certificates or because of a failed TLS negotiation.
I've figured out the issue. I had to take ownership over the SQL Server. After I got on it I realized 2 things.
So I attempted to put a certificate on the SQL Server service however, the service wouldn't start. After some further research, the service won't start if its being run as a local account. I created a network account and gave it the proper permissions and then applied it to the SQL Server Service. Once I did that then I started service successfully and the original error message disappeared.
I am now running into a different issue trying to test the connections however, it looks like I am running into a known issue (https://community.splunk.com/t5/All-Apps-and-Add-ons/Splunk-DBConnect-3-1-1-Build-34-There-was-an-er...)? Can someone confirm? Error message:
There was an error processing your request. It has been logged (ID ddc19c6c869a60ee)
Fixed. Upgraded driver to the 9.4 jre8 driver from Microsoft.