All Apps and Add-ons

Azure Event Hubs Simple Grabber: u'eventhub.pysdk': 3 clients failed to start.

anthonysomerset
Path Finder

I'm trying to set this app up with an event hub, I have verified on my heavy forwarder box that I can use the Azure example python scripts to receive events from the event hub so I have verified connectivity and credentials are OK:

[9:59:58] (ssh) (SUDO) root@prod-backups:~ # python recieve.py        
Received: <azure.eventhub.common.Offset object at 0x7f7ee2456510>, 0
Received: <azure.eventhub.common.Offset object at 0x7f7ee2456610>, 1
Received 2 messages in 0.0514070987701 seconds

However, when I attempt to connect to the exact same event hub with the same credentials from the Splunk app it does not appear to be connecting.

With logs similar to this:

2019-06-21 10:09:47,140 INFO pid=15675 tid=MainThread file=client.py:run:315 | u'eventhub.pysdk-b7c6c13c': Starting 2 clients
2019-06-21 10:09:47,141 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
2019-06-21 10:09:47,587 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
2019-06-21 10:09:47,790 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.START: 0> to <ConnectionState.HDR_SENT: 2>
2019-06-21 10:09:47,840 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.HDR_SENT: 2> to <ConnectionState.HDR_EXCH: 3>
2019-06-21 10:09:47,840 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.HDR_EXCH: 3> to <ConnectionState.OPEN_SENT: 7>
2019-06-21 10:09:47,891 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.OPEN_SENT: 7> to <ConnectionState.OPENED: 9>
2019-06-21 10:09:47,992 INFO pid=15675 tid=MainThread file=connection.py:work:260 | CBS for connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' completed opening with status: 0
2019-06-21 10:09:48,042 INFO pid=15675 tid=MainThread file=connection.py:work:260 | Token put complete with result: 0, status: 202, description: 'Accepted', connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'
2019-06-21 10:09:48,093 INFO pid=15675 tid=MainThread file=receiver.py:on_state_changed:296 | Message receiver 'receiver-link-684250f6-5e18-4efc-9dc6-9e0d584b0597' state changed from <MessageReceiverState.Idle: 0> to <MessageReceiverState.Opening: 1> on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'
2019-06-21 10:09:48,143 INFO pid=15675 tid=MainThread file=receiver.py:on_state_changed:296 | Message receiver 'receiver-link-684250f6-5e18-4efc-9dc6-9e0d584b0597' state changed from <MessageReceiverState.Opening: 1> to <MessageReceiverState.Open: 2> on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'
2019-06-21 10:09:48,194 WARNING pid=15675 tid=MainThread file=client.py:run:324 | u'eventhub.pysdk-b7c6c13c': 1 clients failed to start.
2019-06-21 10:10:18,465 INFO pid=15675 tid=MainThread file=client.py:stop:339 | u'eventhub.pysdk-b7c6c13c': Stopping 2 clients
2019-06-21 10:10:18,465 INFO pid=15675 tid=MainThread file=receiver.py:on_state_changed:296 | Message receiver 'receiver-link-684250f6-5e18-4efc-9dc6-9e0d584b0597' state changed from <MessageReceiverState.Open: 2> to <MessageReceiverState.Closing: 3> on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'
2019-06-21 10:10:18,619 INFO pid=15675 tid=MainThread file=connection.py:_close:130 | Shutting down connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.
2019-06-21 10:10:18,620 INFO pid=15675 tid=MainThread file=cbs_auth.py:close_authenticator:82 | Shutting down CBS session on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.
2019-06-21 10:10:18,620 INFO pid=15675 tid=MainThread file=cbs_auth.py:close_authenticator:86 | Auth closed, destroying session on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.
2019-06-21 10:10:18,620 INFO pid=15675 tid=MainThread file=cbs_auth.py:close_authenticator:89 | Finished shutting down CBS session on connection: 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.
2019-06-21 10:10:18,620 INFO pid=15675 tid=MainThread file=connection.py:_state_changed:178 | Connection 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1' state changed from <ConnectionState.OPENED: 9> to <ConnectionState.END: 13>
2019-06-21 10:10:18,622 INFO pid=15675 tid=MainThread file=connection.py:_close:137 | Connection shutdown complete 'EHReceiver-ab0eed85-5125-4f09-8ff1-84d364014c4f-partition1'.

anthonysomerset
Path Finder

googling for the error itself suggests a firewall issue specifically but as mentioned using an example MS python script works fine with same credentials/endpoint - seems some other error or config breaking?

0 Karma

anthonysomerset
Path Finder

I'd like to add that since this question was published - this app was archived - app author is obviously not interested in supporting it

0 Karma

mitchcorbett
Engager

Did you ever find a solution to this issue? We're getting the same thing. We checked the firewall, and it looks correct. Thanks.

0 Karma

anthonysomerset
Path Finder

no - i had to go and use the Capture based splunk app for now - i think it comes down to the data coming off the event hub must be in a specific format with some specific fields present that at the very least in my case are not present

0 Karma

mitchcorbett
Engager

Thanks for responding. Which is the capture app? We tried using the event hubs integrator app from the same author with no luck.

0 Karma

anthonysomerset
Path Finder

https://splunkbase.splunk.com/app/4343/#/detail

we did some custom tweaks to not hardcode sourcetype, host and index though so that the inputs config could properly override and we could parse the events correctly

0 Karma

mitchcorbett
Engager

Great, thanks for the tip. We'll give it another go.

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...