topic Re: Setting up HEC (HTTP Event Collector) in a indexer cluster in Getting Data In

Setting up HEC (HTTP Event Collector) in a indexer cluster

patng_nw — Tue, 27 Jul 2021 08:10:51 GMT

I am trying to set up HEC for my indexer cluster (v8.0.7), with 2 indexers (and 3 search heads) managed by a master node.

I read multiple docs and articles already, but I want to make sure I get some basic ideas correct first.

In a non-clustered env, it's simple and each HEC client will talk to port 8088 of one indexer.

But in an indexer cluster environment:

Which server will an HEC client talk to?
How can "load balancing of indexing" be achieved?
Is the master node in any way involved?

Re: Setting up HEC (HTTP Event Collector) in a indexer cluster

venkatasri — Tue, 27 Jul 2021 08:34:02 GMT

Hi @patng_nw

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

Re: Setting up HEC (HTTP Event Collector) in a indexer cluster

patng_nw — Tue, 27 Jul 2021 08:48:09 GMT

That's clear now. I hope Splunk can update their doc to at least give us a high level picture.

Re: Setting up HEC (HTTP Event Collector) in a indexer cluster

venkatasri — Tue, 27 Jul 2021 08:57:14 GMT

@patng_nw link Components that help to manage your deployment - Splunk Documentation having some info but not at HEC level. Set up and use HTTP Event Collector in Splunk Web - Splunk Documentation further reading.

Same you can use DS as well push to indexers, if you have HF in distributed set-up that's one of the place HEC can be enabled. It all depends how huge the load is on indexers.

Re: Setting up HEC (HTTP Event Collector) in a indexer cluster

patng_nw — Tue, 27 Jul 2021 09:15:13 GMT

Thanks for the links. I have read them before I post this question here, as they don't provide a clear high-level picture which list out the options we have.