All Apps and Add-ons

Docker splunk-url for cluster: Multiple values for cluster?

wsanderstii
Path Finder

For the the splunk logging driver in Docker that sends container logs to HTTP Event Collector in Splunk Enterprise and Splunk Cloud, is it possible to specify multiple values to send data to a cluster?

I.E.

    ...
    "splunk-url": "https://server1:8089",
    "splunk-url": "https://server2:8089",
    ...

or

    ...
    "splunk-url": "https://server1:8089, https://server2:8089",
    ...
0 Karma

outcoldman
Communicator

That is not supported by Splunk Logging Driver.
You need to setup your own Reverse Proxy (HAProxy or Nginx will work) in front of HEC to do that.

0 Karma

wsanderstii
Path Finder

Thanks.

OTOH the Universal Forwarder does support this, i.e. in outputs.conf:

[tcpout]
defaultGroup = default-autolb-group
[tcpout:default-autolb-group]
server = splunk01:9997,splunk02:9997

And the splunkd will sort-of round robin between the two.

0 Karma

mattymo
Splunk Employee
Splunk Employee

have you looked at the UF as an option instead of the logging driver? What is your motivation for using the driver?

I know that in some scenarios, teams are looking to avoid logging back to the host node (which json driver does), but if that is not a concern for you, I strongly recommend you check out the UF.

I have had much success with the UF as a daemonset (in k8s) or on the docker host while using the default json driver.

Also ensures you get all the features you know and trust from the UF, and I am posting configs here (https://github.com/splunk/docker-itmonitoring) that will help you with multiline log support, indextime field extractions and beyond.

Let me know if you'd like to try the UF out and I can get you set up.

0 Karma

outcoldman
Communicator

@wsanderstii, correct Splunk Forwarder does support this. Any reason why you cannot install Balancer in Front?

On side note. Just want to share, that I am author of Splunk Logging Driver. I left Splunk year ago and co-founded own company, which is focused on Monitoring Container environments in Splunk. We have our own solution for forwarding Logs and Metrics from Docker to Splunk. We do not have this feature yet, but currently we are working on v3, which will include a lot of features in our applications and collector (Indexer Acknowledgment support, Capacity planning monitoring, Security Monitoring and more). It should not be hard for us to implement balancing on Splunk output as well. If you will be interested to give it a try https://www.outcoldsolutions.com and send me email denis@outcoldsolutions.com

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!