Retrieving Docker container logs using Splunk

[nanduni]

Hi all,

I am a newbie to Splunk and since few days, I am attempting to use Splunk to retrieve docker container logs. I tried using docker image of Splunk Enterprise. With that, I could access the Splunk instance on the browser through http://localhost:8000. Thereafter, I am stuck. I used the following command and that gave me the following error message.

docker run --log-driver=splunk --log-opt splunk-url=http://localhost:8000 --log-opt splunk-token=B0AE18EB-4A5F-4A78-911D-033265BA430A nginx
docker: Error response from daemon: Failed to initialize logging driver: splunk: failed to verify connection - 303 See Other - http-equiv="content-type" content="text/html; charset=UTF-8">

[nanduni · Accepted solution]

I managed to resolve this by running Splunk Enterprise Docker Image as follows.

docker run -d -e "SPLUNK_START_ARGS=--accept-license" -e "SPLUNK_USER=root" -p "8000:8000" -p "8088:8088" splunk/splunk

@dcharboneau, Thank you very much for pointing out my errors.

[dcharboneau_spl]

I believe this could be the TCP port you are using is 8000 which is the web interface. Try 8088. Also looks like you need to be using HTTPS not HTTP.

https://docs.docker.com/engine/admin/logging/splunk/#splunk-options

[nanduni]

Thank you for the response.

I tried that as well, still gives me errors.

$ docker run --log-driver=splunk --log-opt splunk-url=https://127.0.0.1:8088 --log-opt splunk-token=FD7C8352-E4FE-40AB-B2EA-01A1DEC6F7D9 nginx
docker: Error response from daemon: Failed to initialize logging driver: dial tcp 127.0.0.1:8088: getsockopt: connection refused.
ERRO[0001] error getting events from daemon: net/http: request canceled

Any suggestions to resolve this ?

Thanks.

[dcharboneau_spl]

Did you try http as well. Ran though HEC setup on splunk and if you didn't select ssl it may just be clear text.

[nanduni]

Yes, I tried that as well. In Global settings, I enabled/disabled SSL and checked with https/http as well. But that still continues to give me the same error log.

In Docker documentation that you have pointed out, it is mentioned to specify the splunk-url in the format https://your_splunk_instance:8088. Here, what does this 'your_splunk_instance' refers to? I assumed that to be 127.0.0.1, from which I accessed the web interface. Am I correct?