,Guacamole Docker logs in Splunk

[tezarin]

Hi,
I would like the Guacamole logs to get forwarded to the Splunk server and I added the log forwarding parameters I found on Splunk docs and ran:
docker inspect -f '{{.HostConfig.LogConfig.Type}}' containerID
and the output was: Splunk,

But I checked on the splunk server, ran the query on the docker host and searched for guacamole, it did not return anything.
[/CODE]
--log-opt splunk-token=******************* \
--log-opt splunk-url=https://splunk aws server:8089 \
--log-opt splunk-insecureskipverify=true \
--log-opt splunk-caname=SplunkServerDefaultCert \
I did the same thing on another host using the same splunk token and was able to see the docker logs on the splunk server.
[/CODE]
Can someone please help me with that?
Thank you
,Hi,

I would like the Guacamole logs to get forwarded to the Splunk server and I added the log forwarding parameters I found on Splunk docs and ran docker inspect -f '{{.HostConfig.LogConfig.Type}}' containerID
and the output was: Splunk, but I checked on the splunk server, ran the query on the docker host and searched for guacamole, it did not return anything.

--log-opt splunk-token=******************* \
--log-opt splunk-url=https://splunk aws server:8089 \
--log-opt splunk-insecureskipverify=true \
--log-opt splunk-caname=SplunkServerDefaultCert \

I did the same thing on another host using the same splunk token and was able to see the docker logs on the splunk server.

Can someone please help me with that?

Thank you

[mattymo]

Hey Tezarin,

Thats Splunk URL looks wrong, 8089 is the management port for Splunk....you want the HTTP Event Collector (HEC) port, which is usually 8088.

Splunk-URL - Path to your Splunk Enterprise, self-service Splunk Cloud instance, or Splunk Cloud managed cluster (including port and scheme used by HTTP Event Collector) in one of the following formats: https://your_splunk_instance:8088 or https://input-prd-p-XXXXXXX.cloud.splunk.com:8088 or https://http-inputs-XXXXXXXX.splunkcloud.com.

https://docs.docker.com/config/containers/logging/splunk/#splunk-options

Also, what version of Docker are you using? We recently released Splunk Connect for Docker, a fully supported update to the logging driver that uses docker's v2 plugin arch and that supersedes the old Splunk logging driver (community supported), but depends on a updated docker engine: https://github.com/splunk/docker-logging-plugin

Be sure to check that out!

[tezarin]

Hi mmodestino,

Thank you for your reply. My docker version on both docker hosts is: Docker version 18.03.1-ce, build 9ee9f40

I have two docker hosts and started forwarding the docker logs on the host1 to Splunk (port 8089 - our admin's advised me to use 8089, not sure why). Docker host 1's logs showed up on Splunk, but docker host 2/s docker logs did not show up. I used the very same token for both and to troubleshoot, I even created a new token for the second host but the logs still are not showing up on the host.

Can you please advise how I can fix this issue? Thank you in advance

[mattymo]

Can you curl the HEC endpoint from the cli of the docker host?

use this to test, just insert your token and/or desired index and correct port to confirm you can reach HEC:

   ```curl -k https://:8088/services/collector -H 'Authorization: Splunk $yourToken ' -d '{"sourcetype": "mysourcetype", "event":"Hello, World!"}' 

```
http://dev.splunk.com/view/event-collector/SP-CAAAE7F

Then we need to talk to your admins, i think....and point them to this plugin - https://github.com/splunk/docker-logging-plugin - as it is the new and improved, docker certified, Splunk supported, open source way to move these logs, and based on your docker version, you should be able to use the plugin as the node default in daemon.json!

You definitely don't normally want to be forwarding over port 8089....I am not sure why that is what you were told, but maybe they have reasons??? if they truly are serving hec on 8089, then try the above command to send an event...also I believe you should check the local docker daemon logs to see if it points to something...

[tezarin]

Thank you.Here is the output for port 8088:

{"text":"Success","code":0}

Port 8089's result:

<msg type="WARN">call not properly authenticated</msg>

[mattymo]

then send to 8088 🙂