All Apps and Add-ons

Does this app support fetching data via proxy?

rvaglid
Explorer

I can not find any information on Splunkbase or in the .conf-files regarding support for using proxy for fetching data.

I notice there is a function "remove_http_proxy_env_vars" in the python code, so I assume if the environment variable http_proxy is set, it is specifically removed when fetching data?

def remove_http_proxy_env_vars():
    for k in ("http_proxy", "https_proxy"):
        if k in os.environ:
            del os.environ[k]
        elif k.upper() in os.environ:
            del os.environ[k.upper()]

Is there a way around this? Is support for proxy in the works for the next version? We have strict rules, forcing us to use proxy when connecting to the Internet.

Cheers
Rolf

0 Karma

pclemo_bw
New Member

Is there an ETA of this proxy functionality being added to this TA?

0 Karma

jconger
Splunk Employee
Splunk Employee

Not yet. But, in the meantime, you can edit the input_module_MS_AAD_audit.py and input_module_MS_AAD_signins.py files. These files do the work for the different inputs. The Python requests library is utilized to make calls to the Microsoft APIs, and this library does support proxy severs. Look for the following lines of code in the files:

header = {'Authorization':'Bearer ' + access_token}
r = requests.get(url,headers=header)

Change it to:

header = {'Authorization':'Bearer ' + access_token}
proxies = {
    'http': 'http://proxy_server_address:proxy_server_port',
    'https': 'http://proxy_server_address:proxy_server_port',
}
r = requests.get(url,proxies=proxies,headers=header)
0 Karma

rvaglid
Explorer

Thanks Jason, seems to work fine. We're running SSL-inspection on the proxies so I received the following, even tho the CA certificates are present in the OS cert store (RedHat 7.4):
2018-04-06 12:20:58,356 ERROR pid=8322 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/modinput_wrapper/base_modinput.py", line 127, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py", line 68, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_audit.py", line 75, in collect_events
r = requests.get(url,proxies=proxies,headers=header)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/requests/adapters.py", line 497, in send
raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:676)

I did a bit of digging and changed the following in ta_ms_aad/modinput_wrapper/base_modinput.py to point to the correct CA store:
return self.rest_helper.send_http_request(url=url, method=method, parameters=parameters, payload=payload,
headers=headers, cookies=cookies, verify='/etc/pki/tls/certs/ca-bundle.crt', cert=cert,
timeout=timeout, proxy_uri=self._get_proxy_uri() if use_proxy else None)

0 Karma

hatalla
Path Finder

hey guys,

I deployed the TA on our Splunk deployment and it is behind a firewall. So I edited both input_module_MS_AAD_audit.py and input_module_MS_AAD_signins.py as such:

proxies = {
'http': 'http://my-proxy-server:3128',
'https':'http://my-proxy-server:3128',
}
r = requests.get(url,proxies=proxies,headers=header)
### r = requests.get(url, headers=header)

But I am seeing this error:

12-10-2018 16:37:11.737 -0500 ERROR ExecProcessor - message from "python /splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py" ERRORcannot concatenate 'str' and 'exceptions.KeyError' objects

12-10-2018 16:37:11.714 -0500 ERROR ExecProcessor - message from "python /splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py" TypeError: cannot concatenate 'str' and 'exceptions.KeyError' objects

12-10-2018 16:37:11.714 -0500 ERROR ExecProcessor - message from "python /splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py" header = {'Accept':'application/json', 'Authorization':'Bearer ' + access_token}

12-10-2018 16:37:11.714 -0500 ERROR ExecProcessor - message from "python /splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py" File "/splunk/app/splunk/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_audit.py", line 90, in get_audit_events

12-10-2018 16:37:11.714 -0500 ERROR ExecProcessor - message from "python /splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py" audit_events = get_audit_events(helper, access_token, url, max_records)

12-10-2018 16:37:11.714 -0500 ERROR ExecProcessor - message from "python /splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py" File "/splunk/app/splunk/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_audit.py", line 129, in collect_events

12-10-2018 16:37:11.714 -0500 ERROR ExecProcessor - message from "python /splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py" input_module.collect_events(self, ew)

12-10-2018 16:37:11.714 -0500 ERROR ExecProcessor - message from "python /splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py" File "/splunk/app/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py", line 72, in collect_events

I've plugged the Azure clientID and secretKey in the TA's config screen. What am I missing?

Thanks.

0 Karma
Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...