Getting Data In

Setting up HEC (HTTP Event Collector) in a indexer cluster

patng_nw
Communicator

I am trying to set up HEC for my indexer cluster (v8.0.7), with 2 indexers (and 3 search heads) managed by a master node.

I read multiple docs and articles already, but I want to make sure I get some basic ideas correct first.

In a non-clustered env, it's simple and each HEC client will talk to port 8088 of one indexer. 

But in an indexer cluster environment:

  • Which server will an HEC client talk to? 
  • How can "load balancing of indexing" be achieved?
  • Is the master node in any way involved?
Labels (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

View solution in original post

0 Karma

patng_nw
Communicator

That's clear now.  I hope Splunk can update their doc to at least give us a high level picture.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@patng_nw  link Components that help to manage your deployment - Splunk Documentation having some info but not at HEC level. Set up and use HTTP Event Collector in Splunk Web - Splunk Documentation further reading.

Same you can use DS as well push to indexers, if you have HF in distributed set-up that's one of the place HEC can be enabled. It all depends how huge the load is on indexers.

0 Karma

patng_nw
Communicator

Thanks for the links.  I have read them before I post this question here, as they don't provide a clear high-level picture which list out the options we have.

 

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

View solution in original post

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!