Solved: Re: Splunk Add-on for Microsoft - SSL Issue when u... - Page 2

brdr · ‎02-14-2023

We upgraded our Splunk Enterprise from v8.2.5 to v9.0.1. When we did, it broke the Add-on for Microsoft 365. Every time a connection is made to microsoft we see this SSL error:

SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1106)')))

Has anyone run into this before?

jp_at_hb · ‎05-08-2023

I had the same issues at first and have concluded that something changed with the python libraries between 8.x and 9.x. I have not done an intense review.. YET. But This is what I found.

CA certs in the following locations are ignored.

3rdparty/certifi/cacert.pem
lib/certifi/cacert.pem

The first just never gets accessed anywhere in the code.

[splunk_ta_o365]$ find . -name \*.py | xargs -ifname grep -iH 3rdparty fname
./bin/splunk_ta_o365_rest_handlers.py:# Adding 3rdparty folder in sys.path for using future module

And the second does not even get checked. Since the certifi lib is not getting included. I created this test to see. By using one of the existing o365 mod inputs.

from splunk_ta_o365_bootstrap import setup_python_path, run_module
import sys

if __name__ == "__main__":
    try:
        setup_python_path()
        import certifi
        import requests
        print('PATH = ' + ":".join(sys.path))
        print('CAcert = '+ certifi.where())
        print('Checking connection to MS...')
        test = requests.get('https://login.microsoftonline.com')
        print('Connection to MS.')
    except requests.exceptions.SSLError as err:
        print('SSL Error. ' + err)

Run it as below. If it fails, it means that you do not have the needed certs.

[splunk_ta_o365]$ splunk cmd python3 bin/testssl.py
PATH = /opt/splunk/etc/apps/splunk_ta_o365/bin:/opt/splunk/etc/apps/splunk_ta_o365/lib:/opt/splunk/etc/apps/splunk_ta_o365/bin:/opt/splunk/etc/apps/splunk_ta_o365/bin:/opt/splunk/lib/python37.zip:/opt/splunk/lib/python3.7:/opt/splunk/lib/python3.7/lib-dynload:/opt/splunk/lib/python3.7/site-packages:/opt/splunk/lib/python3.7/site-packages/bottle-0.12.19-py3.7.egg
CAcert = /opt/splunk/lib/python3.7/site-packages/certifi/cacert.pem
Checking connection to MS...
Connection to MS.

It is the system CA cert file that is getting read. If you add your proxy certs, etc to the bottom of it. It should work.

I have not looked for the exact reason as of yet, nor filed a ticket yet. This was just a work around and NOT the way to do this.

I hope it helps.
Cheers.

View solution in original post

SinghK · ‎03-09-2023

I have not found any problems with v9 at all and i have o365 data integration. so cant comment on that you need to check logs. if still cant find anything open a case with support.

Bazza_12 · ‎03-09-2023

Our CA signed cert

SinghK · ‎03-09-2023

As i was saying check if the inputs have the correct path to cert and error is very specific

"verify failed: self signed certificate in certificate chain"

inputs are detecting a self signed cert instead of CA one.

SinghK · ‎03-09-2023

check the cert with this :

$SPLUNK_HOME/bin/splunk cmd openssl x509 -enddate -noout -in file.cer

make sure your $SPLUNK_HOME is set correctly or just set the path replacing $SPLUNK_HOME

Bazza_12 · ‎03-09-2023

I can verify that the cert in the apps locations is valid

Bazza_12 · ‎03-09-2023

Same here interested in any feedback

Splunk Add-on for Microsoft - SSL Issue when upgrading to enterprise v9?

configuration

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Announcing Modern Navigation: A New Era of Splunk User Experience

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

Join the Conversation