All Apps and Add-ons

Azure Event Hubs Simple Grabber: u'eventhub.pysdk': 3 clients failed to start.

anthonysomerset
Path Finder

I'm trying to set this app up with an event hub, I have verified on my heavy forwarder box that I can use the Azure example python scripts to receive events from the event hub so I have verified connectivity and credentials are OK:

[9:59:58] (ssh) (SUDO) root@prod-backups:~ # python recieve.py        
Received: <azure.eventhub.common.Offset object at 0x7f7ee2456510>, 0
Received: <azure.eventhub.common.Offset object at 0x7f7ee2456610>, 1
Received 2 messages in 0.0514070987701 seconds

However, when I attempt to connect to the exact same event hub with the same credentials from the Splunk app it does not appear to be connecting.

With logs similar to this:

2019-06-21 10:09:47,140 INFO pid=15675 tid=MainThread file=client.py:run:315 | u'eventhub.pysdk-b7c6c13c': Starting 2 clients
2019-06-21 10:09:47,141 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
2019-06-21 10:09:47,587 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
2019-06-21 10:09:47,790 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.START: 0> to <ConnectionState.HDR_SENT: 2>
2019-06-21 10:09:47,840 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.HDR_SENT: 2> to <ConnectionState.HDR_EXCH: 3>
2019-06-21 10:09:47,840 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.HDR_EXCH: 3> to <ConnectionState.OPEN_SENT: 7>
2019-06-21 10:09:47,891 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.OPEN_SENT: 7> to <ConnectionState.OPENED: 9>
2019-06-21 10:09:47,992 INFO pid=15675 tid=MainThread file=connection.py:work:260 | CBS for connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' completed opening with status: 0
2019-06-21 10:09:48,042 INFO pid=15675 tid=MainThread file=connection.py:work:260 | Token put complete with result: 0, status: 202, description: 'Accepted', connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'
2019-06-21 10:09:48,093 INFO pid=15675 tid=MainThread file=receiver.py:on_state_changed:296 | Message receiver 'receiver-link-684250f6-5e18-4efc-9dc6-9e0d584b0597' state changed from <MessageReceiverState.Idle: 0> to <MessageReceiverState.Opening: 1> on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'
2019-06-21 10:09:48,143 INFO pid=15675 tid=MainThread file=receiver.py:on_state_changed:296 | Message receiver 'receiver-link-684250f6-5e18-4efc-9dc6-9e0d584b0597' state changed from <MessageReceiverState.Opening: 1> to <MessageReceiverState.Open: 2> on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'
2019-06-21 10:09:48,194 WARNING pid=15675 tid=MainThread file=client.py:run:324 | u'eventhub.pysdk-b7c6c13c': 1 clients failed to start.
2019-06-21 10:10:18,465 INFO pid=15675 tid=MainThread file=client.py:stop:339 | u'eventhub.pysdk-b7c6c13c': Stopping 2 clients
2019-06-21 10:10:18,465 INFO pid=15675 tid=MainThread file=receiver.py:on_state_changed:296 | Message receiver 'receiver-link-684250f6-5e18-4efc-9dc6-9e0d584b0597' state changed from <MessageReceiverState.Open: 2> to <MessageReceiverState.Closing: 3> on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'
2019-06-21 10:10:18,619 INFO pid=15675 tid=MainThread file=connection.py:_close:130 | Shutting down connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.
2019-06-21 10:10:18,620 INFO pid=15675 tid=MainThread file=cbs_auth.py:close_authenticator:82 | Shutting down CBS session on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.
2019-06-21 10:10:18,620 INFO pid=15675 tid=MainThread file=cbs_auth.py:close_authenticator:86 | Auth closed, destroying session on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.
2019-06-21 10:10:18,620 INFO pid=15675 tid=MainThread file=cbs_auth.py:close_authenticator:89 | Finished shutting down CBS session on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.
2019-06-21 10:10:18,620 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.OPENED: 9> to <ConnectionState.END: 13>
2019-06-21 10:10:18,622 INFO pid=15675 tid=MainThread file=connection.py:_close:137 | Connection shutdown complete 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.

anthonysomerset
Path Finder

googling for the error itself suggests a firewall issue specifically but as mentioned using an example MS python script works fine with same credentials/endpoint - seems some other error or config breaking?

0 Karma

anthonysomerset
Path Finder

I'd like to add that since this question was published - this app was archived - app author is obviously not interested in supporting it

0 Karma

mitchcorbett
Engager

Did you ever find a solution to this issue? We're getting the same thing. We checked the firewall, and it looks correct. Thanks.

0 Karma

anthonysomerset
Path Finder

no - i had to go and use the Capture based splunk app for now - i think it comes down to the data coming off the event hub must be in a specific format with some specific fields present that at the very least in my case are not present

0 Karma

mitchcorbett
Engager

Thanks for responding. Which is the capture app? We tried using the event hubs integrator app from the same author with no luck.

0 Karma

anthonysomerset
Path Finder

https://splunkbase.splunk.com/app/4343/#/detail

we did some custom tweaks to not hardcode sourcetype, host and index though so that the inputs config could properly override and we could parse the events correctly

0 Karma

mitchcorbett
Engager

Great, thanks for the tip. We'll give it another go.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...