All Apps and Add-ons

Splunk Add-on for Microsoft - SSL Issue when upgrading to enterprise v9?

brdr
Contributor

We upgraded our Splunk Enterprise from v8.2.5 to v9.0.1.  When we did, it broke the Add-on for Microsoft 365. Every time a connection is made to microsoft we see this SSL error:

SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1106)')))

Has anyone run into this before?

 

 

Labels (1)
Tags (1)
1 Solution

jp_at_hb
Explorer

I had the same issues at first and have concluded that something changed with the python libraries between 8.x and 9.x.  I have not done an intense review.. YET. But This is what I found.

CA certs in the following locations are ignored.

3rdparty/certifi/cacert.pem
lib/certifi/cacert.pem

The first just never gets accessed anywhere in the code.

[splunk_ta_o365]$ find . -name \*.py | xargs -ifname grep -iH 3rdparty fname
./bin/splunk_ta_o365_rest_handlers.py:# Adding 3rdparty folder in sys.path for using future module

And the second does not even get checked. Since the certifi lib is not getting included. I created this test to see. By using one of the existing o365 mod inputs. 

from splunk_ta_o365_bootstrap import setup_python_path, run_module
import sys

if __name__ == "__main__":
    try:
        setup_python_path()
        import certifi
        import requests
        print('PATH = ' + ":".join(sys.path))
        print('CAcert = '+ certifi.where())
        print('Checking connection to MS...')
        test = requests.get('https://login.microsoftonline.com')
        print('Connection to MS.')
    except requests.exceptions.SSLError as err:
        print('SSL Error. ' + err)

 

Run it as below. If it fails, it means that you do not have the needed certs.

[splunk_ta_o365]$ splunk cmd python3 bin/testssl.py
PATH = /opt/splunk/etc/apps/splunk_ta_o365/bin:/opt/splunk/etc/apps/splunk_ta_o365/lib:/opt/splunk/etc/apps/splunk_ta_o365/bin:/opt/splunk/etc/apps/splunk_ta_o365/bin:/opt/splunk/lib/python37.zip:/opt/splunk/lib/python3.7:/opt/splunk/lib/python3.7/lib-dynload:/opt/splunk/lib/python3.7/site-packages:/opt/splunk/lib/python3.7/site-packages/bottle-0.12.19-py3.7.egg
CAcert = /opt/splunk/lib/python3.7/site-packages/certifi/cacert.pem
Checking connection to MS...
Connection to MS.

It is the system CA cert file that is getting read. If you add your proxy certs, etc to the bottom of it. It should work. 

I have not looked for the exact reason as of yet, nor filed a ticket yet. This was just a work around and NOT the way to do this. 

I hope it helps. 
Cheers.

View solution in original post

SinghK
Builder

I have not found any problems with v9 at all and i have o365 data integration. so cant comment on that you need to check logs. if still cant find anything open a case with support.

0 Karma

Bazza_12
Path Finder

Our CA signed cert 

0 Karma

SinghK
Builder

As i was saying check if the inputs have the correct path to cert and error is very specific 

"verify failed: self signed certificate in certificate chain"

inputs are detecting a self signed cert instead of CA one.

0 Karma

SinghK
Builder

check the cert with this :

$SPLUNK_HOME/bin/splunk cmd openssl x509 -enddate -noout -in file.cer

make sure your $SPLUNK_HOME is set correctly or just set the path replacing $SPLUNK_HOME

0 Karma

Bazza_12
Path Finder

I can verify that the cert in the apps locations is valid 

 

0 Karma

Bazza_12
Path Finder

Same here interested in any feedback

0 Karma
Get Updates on the Splunk Community!

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...

Integrating Splunk Search API and Quarto to Create Reproducible Investigation ...

 Splunk is More Than Just the Web Console For Digital Forensics and Incident Response (DFIR) practitioners, ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...