I've installed the TA on our Heavy Forwarder, and configured it with the details needed to connect to the Event Hub, as well as the settings for our proxy that it needs to use.
Despite this, we're not seeing any traffic on our proxy from the Heavy Forwarder, despite it appearing to have tried and failed to connect:
2020-03-30 10:37:53,193 ERROR pid=11042 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/modinput_wrapper/base_modinput.py", line 127, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/azure_event_hub.py", line 92, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/input_module_azure_event_hub.py", line 112, in collect_events
partition_ids = client.get_partition_ids()
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/azure/eventhub/client.py", line 163, in get_partition_ids
return self.get_properties()['partition_ids']
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/azure/eventhub/client.py", line 146, in get_properties
response = self._management_request(mgmt_msg, op_type=b'com.microsoft:eventhub')
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/azure/eventhub/client.py", line 127, in _management_request
self._handle_exception(exception, retry_count, max_retries)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/azure/eventhub/client.py", line 105, in _handle_exception
_handle_exception(exception, retry_count, max_retries, self)
File "/opt/splunk/etc/apps/TA-MS-AAD/bin/azure/eventhub/error.py", line 196, in _handle_exception
raise error
ConnectError: Unable to open management session. Please confirm URI namespace exists.
Unable to open management session. Please confirm URI namespace exists.
I've configured the proxy and can see it showing up in ta_ms_aad_settings.conf as:
[proxy]
proxy_enabled = 1
proxy_port = <proxyport>
proxy_url = <proxyurl>
And when searching for sourcetype=ta:ms:aad:log, I can see the message:
2020-03-30 10:37:58,475 DEBUG pid=12915 tid=MainThread file=base_modinput.py:log_debug:286 | _Splunk_ Proxy is enabled: <proxyurl>:<proxyport>
However when I run tcpdump on the host, I can see it making DNS requests to resolve the host in the EventHub Connection String I provided it, and then making a request directly out to the host - without using the proxy
2020-03-27 15:34:38.750364 IP (tos 0x0, ttl 64, id 26550, offset 0, flags [DF], proto TCP (6), length 60)
<SplunkHeavyForwarder> > <AzureEventHub>: Flags [S], cksum 0xf2b6 (incorrect -> 0xbf5a), seq 330547664, win 29200, options [mss 1460,sackOK,TS val 3370165862 ecr 0,nop,wscale 7], length 0
2020-03-27 15:34:39.752354 IP (tos 0x0, ttl 64, id 26551, offset 0, flags [DF], proto TCP (6), length 60)
<SplunkHeavyForwarder> > <AzureEventHub>: Flags [S], cksum 0xf2b6 (incorrect -> 0xbb70), seq 330547664, win 29200, options [mss 1460,sackOK,TS val 3370166864 ecr 0,nop,wscale 7], length 0
2020-03-27 15:34:41.757349 IP (tos 0x0, ttl 64, id 26552, offset 0, flags [DF], proto TCP (6), length 60)
<SplunkHeavyForwarder> > <AzureEventHub>: Flags [S], cksum 0xf2b6 (incorrect -> 0xb39b), seq 330547664, win 29200, options [mss 1460,sackOK,TS val 3370168869 ecr 0,nop,wscale 7], length 0
2020-03-27 15:34:45.768341 IP (tos 0x0, ttl 64, id 26553, offset 0, flags [DF], proto TCP (6), length 60)
<SplunkHeavyForwarder> > <AzureEventHub>: Flags [S], cksum 0xf2b6 (incorrect -> 0xa3f0), seq 330547664, win 29200, options [mss 1460,sackOK,TS val 3370172880 ecr 0,nop,wscale 7], length 0
I'm out of ideas on where this is failing - has anyone had a similar issue, or can you see something I've missed?
I don't need to edit ta_ms_aad_settings.conf.spec do I? I assume it's like a template for the ta_ms_aad_settings.conf which has been populated with the proxy config. Currently the spec file is empty:
[proxy]
proxy_enabled =
proxy_type =
proxy_url =
proxy_port =
proxy_username =
proxy_password =
proxy_rdns =
[logging]
loglevel =
... View more