attempts to get data fail due to SSL error

[pdoconnell]

In /opt/splunk/var/log/splunk/ta_gitlab_add_on_get_events.log I see this error on attempts to get logs from my gitlab instance:
2019-03-09 22:50:44,802 ERROR pid=13454 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/modinput_wrapper/base_modinput.py", line 127, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/get_events.py", line 68, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/input_module_get_events.py", line 297, in collect_events
headers=headers)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/modinput_wrapper/base_modinput.py", line 476, in send_http_request
proxy_uri=self._get_proxy_uri() if use_proxy else None)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/splunk_aoblib/rest_helper.py", line 43, in send_http_request
return self.http_session.request(method, url, **requests_args)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/requests/adapters.py", line 497, in send
raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:741)

This appears to be from the TLS certificate for my gitlab instance coming from my internal CA. How would I either tell this TA to ignore SSLErrors, or tell the TA to trust the CA?

[ledan9286]

the workaround i have for this is as follows:

1) Go to the following directory /opt/splunk/etc/apps/TA-gitlab-add-on/bin/ta_gitlab_add_on/modinput_wrapper

2) vi base_modinput.py

3) Find line number 456

4) Change the verify=True to verify=False (Case sensitive)

5) save and quit. It should work

[nickhills]

Hi @pdoconnell,

You have two options:
1.) Tell Splunk Ignore the error
2.) Tell Splunk to trust your CA.

The latter is the 'better' approach but both of them require a code change in the python scripts.

To fix(hack) 1, I would start in adaptors.py and work backwards to find the SSL CA verification process - I would expect there to be a switch with a default value 'verify_ca=true' or similar (I haven't checked) you could try flipping this to false.

To fix 2 take a look here: https://stackoverflow.com/questions/39356413/how-to-add-a-custom-ca-root-certificate-to-the-ca-store...
You need your CA to the SSL_CERT_DIR which python uses - you may need to create a folder for your root CA, and set it in the ENV

I would strongly advise against adding your CA to the Splunk trusted certs as these will likely get overwritten with each update.

Ideally the vendor of the TA would include support to ignore cert checking/set a path to check for local root CA Certs.