Dynatrace Addon for Splunk unable to communicate and get logs

[kvm]

Hi,
I’m trying to integrate dynatrace with Splunk using Dynatrace add-on for Splunk.

However after the configuration, I’m getting the below error.

(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1143)'))),

Anyone experienced this, or know how to solve this certificate issue?

FYI, I have updated ssl certificate on both Splunk and Dynatrace, but it didn’t help.

[Meett]

Can you try to add SSL CA Chain to below location and see if it works?

1) /opt/splunk/lib/python3.7/site-packages/certifi

And

2) /etc/apps/<Add-on_folder>/lib/certify

[kvm]

Hi @Meett ,

Can you try to add SSL CA Chain to below location  - By this do you mean Dynatrace SSL certificate to update in the Splunk Add-on ? Please confirm.

[Meett]

Hey @kvm , 

Not an Dynatrace SSL, but from ERRORs you added it looks like at network level its causing some issue, if you have CACert chain configured at Splunk level then you should add same Cert to locations i mentioned.

[kvm]

Hi @Meett 

I did update the SSL certificate Splunk is using which is basically the combination of server cert + intermediate cert + root CA cert.

But still same errors.

[PickleRick]

That's not how it works. I don't know this particular solution but I'm assuming it's trying to connect to some Dynatrace API endpoint to retrieve data.

For this it has to verify its certificate. Not yours. You don't use your Splunk certificate here nor you use the certificate of the CA that issued your Splunk's certificate. For the verification to work you must trust the RootCA which is in the root of the certification path of the server providing the Dynatrace API.

[inventsekar]

Hi @kvm 

More details needed pls:

1) Splunk version,

2) Cloud or on-prim,

3) Dynatrace version,

4) UF (and HF) version?

5) Splunk's own SSL Certificate(Linux's SSL certificate) or third party SSL certificate?

[PickleRick]

Also be sure that you're not using any TLS-inspection solution if it's on-prem.

[inventsekar]

Hi @PickleRick 

The SSL is the place users get more difficult/time-consuming issues and the least help. 

the Splunk documentations SSL Certs is good one, but still looks like something missing. 

may i know if you are aware of any Splunk Conf's discussions around the SSL pls, thanks. 

Best Regards

Sekar

[PickleRick]

No, I'm not aware of .conf talks about TLS, unfortunately.

The problem with TLS is that it might be a quite complicated issue on its own even without Splunk on top of it.