enabling HTTP Event Collector for docker container on splunk clustered setup

[sim_tcr]

We tested HTTP Event Collector for a docker container by starting the container with below,

--log-driver=splunk --log-opt splunk-token=<token> --log-opt splunk-url=https://<splunkserver>:8088 --log-opt splunk-insecureskipverify=true

Now we want to send the events to our actual splunk setup. We are on search head clustering and index clustering enabled, with a separate shc deployer, deployment server and index master servers.

• Which server we should enable the HTTP Event Collector?
• How can i specify the index where the events should be forwarded?

Thanks,

[outcoldman]

Hi @sim_tcr,

If you are just integrating Docker with Splunk, have you seen our solution for collecting logs and metrics https://splunkbase.splunk.com/app/3723/? You can find some documentation https://www.outcoldsolutions.com/docs/ demos and screencasts. We have also comparison table with official Splunk Logging Driver https://www.outcoldsolutions.com/docs/collectorfordocker/#comparing-with-splunk-logging-driver (btw, I am the original author of this driver).

[mattymo]

Hi @sim_tcr!

The best advice for you would require a peek at your existing architecture and data volumes, but you should start by having a look at dev.splunk.com for HEC architectures:

http://dev.splunk.com/view/event-collector/SP-CAAAE73

If you are just getting started you could look at having a Heavy forwarder in place to catch the HEC traffic and pass it to your indexers, or a pool of HFs, or you could also just enable it on all your indexers - although you will want to be careful with that depending on the volume of traffic you expect.

As for routing to an index, check out the docs on HEC tokens. You can set the index/sourcetype there!

http://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector

[sim_tcr]

Thank you for responding.
I think we want to go with http://dev.splunk.com/view/event-collector/SP-CAAAE73#scen1
We are at 6.3.3 and referring http://docs.splunk.com/Documentation/Splunk/6.3.3/Data/UsetheHTTPEventCollector
when checking "select allowed indexes" field, we do not see all our indexes from the index cluster listed there.
Do you know what are we missing?

[sloshburch]

Great answer @mmodestino!

@sim_tcr - Remember that if you start with Scenario 1 and you have to change/rebuilt the HEC server, you'll need to update all the clients (apps) sending data. As such, use a VIP or a Load Balancer even if pointing to that one instance. This will increase availability and failover options besides allowing for scalability later and essentially making the transition to Scenario 3 seamless.

[mattymo]

You need to tell your HF about them in indexes.conf

Grab the TA from the indexer cluster and use that indexes.conf on the HF.