Getting Data In

Setting up HEC (HTTP Event Collector) in a indexer cluster

patng_nw
Communicator

I am trying to set up HEC for my indexer cluster (v8.0.7), with 2 indexers (and 3 search heads) managed by a master node.

I read multiple docs and articles already, but I want to make sure I get some basic ideas correct first.

In a non-clustered env, it's simple and each HEC client will talk to port 8088 of one indexer. 

But in an indexer cluster environment:

  • Which server will an HEC client talk to? 
  • How can "load balancing of indexing" be achieved?
  • Is the master node in any way involved?
Labels (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

View solution in original post

0 Karma

patng_nw
Communicator

That's clear now.  I hope Splunk can update their doc to at least give us a high level picture.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@patng_nw  link Components that help to manage your deployment - Splunk Documentation having some info but not at HEC level. Set up and use HTTP Event Collector in Splunk Web - Splunk Documentation further reading.

Same you can use DS as well push to indexers, if you have HF in distributed set-up that's one of the place HEC can be enabled. It all depends how huge the load is on indexers.

0 Karma

patng_nw
Communicator

Thanks for the links.  I have read them before I post this question here, as they don't provide a clear high-level picture which list out the options we have.

 

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

0 Karma
Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...