All Apps and Add-ons

Splunk TA for Okta stopped working with ServerNotFoundError

Explorer

A few weeks ago our Okta feed to Splunk stopped working. I have tried everything I can think of to fix this issue. It seems to be related to a socket error based on the built in python script that comes with the TA. See below from the ta_okta.log

I have tried removing and recreating the data inputs, changing the start time stamp, modifying the python script to increase the timeout, removing and reinstalling the TA and the host can indeed be resolved with nslookup from the heavy forwarder where this TA is installed. Also had a new API key generated and added it to the data inputs.

Any help would be greatly appreciated!

bash-4.1$ tail ta_okta.log
File "/scratch/splunk/etc/apps/Splunk_TA_okta/bin/okta_rest_client.py", line 80, in request
headers=headers)
File "/scratch/splunk/etc/apps/Splunk_TA_okta/bin/splunktalib/httplib2/init.py", line 1593, in request
(response, content) = self.request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/scratch/splunk/etc/apps/Splunk_TA_okta/bin/splunktalib/httplib2/
init.py", line 1335, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/scratch/splunk/etc/apps/Splunk_TA_okta/bin/splunktalib/httplib2/
init_.py", line 1263, in _conn_request
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)
ServerNotFoundError: Unable to find the server at company.okta.com

This is from the referenced python script for the Okta TA.

def _conn_request(self, conn, request_uri, method, body, headers):
i = 0
seen_bad_status_line = False
while i < RETRIES:
i += 1
try:
if hasattr(conn, 'sock') and conn.sock is None:
conn.connect()
conn.request(method, request_uri, body, headers)
except socket.timeout:
raise
except socket.gaierror:
conn.close()
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)

0 Karma

Explorer

It turns out this was a DNS issue. When I first saw the ServerNotFound error in the logs I tried an nslookup and ping of the Okta host and they both worked so I assumed it was some other issue. But after later trying a curl to the host (like mentioned above by hozhang) it gave an unknown host error. So in doing some more searching it turns out that it is possible for nslookup to work while curl does not for DNS resolution in certain cases. So I decided to modify my /etc/resolv.conf and add a different name server and it worked!

The end issue was one of the name servers in the list at the top was failing.

Thanks for the help!

0 Karma

Splunk Employee
Splunk Employee

According to the log, it seems the server host is not reachable. Would you try the following command to confirm it?

curl -i  -H "Accept:application/json" -H "Content-Type:application/json" -H "Authorization:<YOUR TOKEN>" https://<YOUR Okta server host>/api/v1/users/me

Thanks!

Path Finder

Hozhang! Thanks for posting! this help me to isolate and issue.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!