All Apps and Add-ons
Highlighted

Splunk TA for Okta stopped working with ServerNotFoundError

Explorer

A few weeks ago our Okta feed to Splunk stopped working. I have tried everything I can think of to fix this issue. It seems to be related to a socket error based on the built in python script that comes with the TA. See below from the ta_okta.log

I have tried removing and recreating the data inputs, changing the start time stamp, modifying the python script to increase the timeout, removing and reinstalling the TA and the host can indeed be resolved with nslookup from the heavy forwarder where this TA is installed. Also had a new API key generated and added it to the data inputs.

Any help would be greatly appreciated!

bash-4.1$ tail taokta.log
File "/scratch/splunk/etc/apps/Splunk
TAokta/bin/oktarestclient.py", line 80, in request
headers=headers)
File "/scratch/splunk/etc/apps/Splunk
TAokta/bin/splunktalib/httplib2/init.py", line 1593, in request
(response, content) = self.
request(conn, authority, uri, requesturi, method, body, headers, redirections, cachekey)
File "/scratch/splunk/etc/apps/Splunk
TAokta/bin/splunktalib/httplib2/init.py", line 1335, in _request
(response, content) = self.
connrequest(conn, requesturi, method, body, headers)
File "/scratch/splunk/etc/apps/SplunkTAokta/bin/splunktalib/httplib2/init.py", line 1263, in connrequest
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)
ServerNotFoundError: Unable to find the server at company.okta.com

This is from the referenced python script for the Okta TA.

def connrequest(self, conn, requesturi, method, body, headers):
i = 0
seen
badstatusline = False
while i < RETRIES:
i += 1
try:
if hasattr(conn, 'sock') and conn.sock is None:
conn.connect()
conn.request(method, request_uri, body, headers)
except socket.timeout:
raise
except socket.gaierror:
conn.close()
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)

0 Karma
Highlighted

Re: Splunk TA for Okta stopped working with ServerNotFoundError

Splunk Employee
Splunk Employee

According to the log, it seems the server host is not reachable. Would you try the following command to confirm it?

curl -i  -H "Accept:application/json" -H "Content-Type:application/json" -H "Authorization:<YOUR TOKEN>" https://<YOUR Okta server host>/api/v1/users/me

Thanks!

Highlighted

Re: Splunk TA for Okta stopped working with ServerNotFoundError

Explorer

Hozhang! Thanks for posting! this help me to isolate and issue.

0 Karma
Highlighted

Re: Splunk TA for Okta stopped working with ServerNotFoundError

Explorer

It turns out this was a DNS issue. When I first saw the ServerNotFound error in the logs I tried an nslookup and ping of the Okta host and they both worked so I assumed it was some other issue. But after later trying a curl to the host (like mentioned above by hozhang) it gave an unknown host error. So in doing some more searching it turns out that it is possible for nslookup to work while curl does not for DNS resolution in certain cases. So I decided to modify my /etc/resolv.conf and add a different name server and it worked!

The end issue was one of the name servers in the list at the top was failing.

Thanks for the help!

0 Karma