All Apps and Add-ons

attempts to get data fail due to SSL error

pdoconnell
Path Finder

In /opt/splunk/var/log/splunk/ta_gitlab_add_on_get_events.log I see this error on attempts to get logs from my gitlab instance:
2019-03-09 22:50:44,802 ERROR pid=13454 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/modinput_wrapper/base_modinput.py", line 127, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/get_events.py", line 68, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/input_module_get_events.py", line 297, in collect_events
headers=headers)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/modinput_wrapper/base_modinput.py", line 476, in send_http_request
proxy_uri=self._get_proxy_uri() if use_proxy else None)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/splunk_aoblib/rest_helper.py", line 43, in send_http_request
return self.http_session.request(method, url, **requests_args)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/requests/adapters.py", line 497, in send
raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:741)

This appears to be from the TLS certificate for my gitlab instance coming from my internal CA. How would I either tell this TA to ignore SSLErrors, or tell the TA to trust the CA?

Tags (1)
0 Karma

ledan9286
Loves-to-Learn

the workaround i have for this is as follows:

1) Go to the following directory /opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/modinput_wrapper

2) vi base_modinput.py

3) Find line number 456

4) Change the verify=True to verify=False (Case sensitive)

5) save and quit. It should work

0 Karma

nickhills
Ultra Champion

Hi @pdoconnell,

You have two options:
1.) Tell Splunk Ignore the error
2.) Tell Splunk to trust your CA.

The latter is the 'better' approach but both of them require a code change in the python scripts.

To fix(hack) 1, I would start in adaptors.py and work backwards to find the SSL CA verification process - I would expect there to be a switch with a default value 'verify_ca=true' or similar (I haven't checked) you could try flipping this to false.

To fix 2 take a look here: https://stackoverflow.com/questions/39356413/how-to-add-a-custom-ca-root-certificate-to-the-ca-store...
You need your CA to the SSL_CERT_DIR which python uses - you may need to create a folder for your root CA, and set it in the ENV

I would strongly advise against adding your CA to the Splunk trusted certs as these will likely get overwritten with each update.

Ideally the vendor of the TA would include support to ignore cert checking/set a path to check for local root CA Certs.

If my comment helps, please give it a thumbs up!
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...