topic Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup? in All Apps and Add-ons

How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

davietch — Wed, 30 Sep 2020 04:47:28 GMT

Hi,

I am setting up the Deep Learning Toolkit (https://splunkbase.splunk.com/app/4607/) but the connexion to the docker instance does not work.

What I did:
- Install docker and add the containers needed
- Docker runs as root
- Install Splunk + Machine Learning Toolkit (+ python maths librairies) + Deep Learning toolkit
- The user running Splunk belongs to the docker group

When I fill in the form in the set up page of the app with the information about docker (single-instance type), I get this error:
UnixHTTPConnectionPool(host='localhost', port=None): Max retries exceeded with url: /v1.35/_ping (Caused by : [Errno 13] Permission denied)

In the internal logs, I also see logs like this:

ConnectionError: UnixHTTPConnectionPool(host='localhost', port=None): Max retries exceeded with url: /v1.35/containers/json?size=0&filters=%7B%22label%22%3A+%5B%22mltk_container%22%5D%7D&limit=-1&all=0&trunc_cmd=0 (Caused by : [Errno 13] Permission denied)

Can you help me to make this work?

Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

PavelP — Mon, 30 Mar 2020 15:28:51 GMT

what is the host OS? CentOS/Ubuntu? Try to disable SELinux/Apparmor.

Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

davietch — Tue, 31 Mar 2020 07:21:10 GMT

Hi,

My OS is RedHat 7.7

SELinux is already disabled:

[aaa@XXXX~]$ sestatus
SELinux status:                 disabled

I don't see any Apparmor software installed.

Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

pdrieger_splunk — Tue, 31 Mar 2020 13:19:57 GMT

Hi @davietch, can you please cross check this answer and see if connecting to tcp instead of the unix://var/run/docker.sock helps in your case, too: https://answers.splunk.com/answers/811981/deep-learning-toolkit-setup.html

Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

davietch — Tue, 31 Mar 2020 14:25:11 GMT

Hi,

No, it didn't work:

With Localhost:

HTTPConnectionPool(host='localhost', port=2375): Max retries exceeded with url: /v1.35/_ping (Caused by : [Errno 111] Connection refused)
With the name of the server:

HTTPConnectionPool(host='myservername', port=2375): Max retries exceeded with url: /v1.35/_ping (Caused by : [Errno 111] Connection refused)

Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

pdrieger_splunk — Tue, 31 Mar 2020 15:33:17 GMT

Hi @davietch , to me it looks like your docker host can not be reached from the DLTK which tries to establish a connection but fails.

for the last case: Can you cross check with this post here if you haven't: https://github.com/docker/docker-py/issues/290
for your initial connection, assuming your docker connection was defined this way: Docker Host: unix://var/run/docker.sock Endpoint URL: localhost External URL: localhost Can you double check that your docker deamon socket is enabled: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-socket-option

Let us know if it can be resolved with that.

Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

rfujara_splunk — Tue, 31 Mar 2020 20:02:22 GMT

Could you please check this Docker docs page:
https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user

It may be because the user running Splunk is not allowed to talk to Docker.

Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

davietch — Thu, 02 Apr 2020 10:12:26 GMT

Turns out the permissions were OK but I needed a login/logout to apply the permissions.. Just that.

Thanks all !

Re: How can I get rid of "Connection refused" for Deep Learning Toolkit setup?

ephemeric — Mon, 23 Aug 2021 15:30:42 GMT

As usual always check SELinux and perms on `/var/run/docker.sock`.