Getting Data In

Setting up HEC (HTTP Event Collector) in a indexer cluster

patng_nw
Communicator

I am trying to set up HEC for my indexer cluster (v8.0.7), with 2 indexers (and 3 search heads) managed by a master node.

I read multiple docs and articles already, but I want to make sure I get some basic ideas correct first.

In a non-clustered env, it's simple and each HEC client will talk to port 8088 of one indexer. 

But in an indexer cluster environment:

  • Which server will an HEC client talk to? 
  • How can "load balancing of indexing" be achieved?
  • Is the master node in any way involved?
Labels (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

View solution in original post

patng_nw
Communicator

That's clear now.  I hope Splunk can update their doc to at least give us a high level picture.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@patng_nw  link Components that help to manage your deployment - Splunk Documentation having some info but not at HEC level. Set up and use HTTP Event Collector in Splunk Web - Splunk Documentation further reading.

Same you can use DS as well push to indexers, if you have HF in distributed set-up that's one of the place HEC can be enabled. It all depends how huge the load is on indexers.

0 Karma

patng_nw
Communicator

Thanks for the links.  I have read them before I post this question here, as they don't provide a clear high-level picture which list out the options we have.

 

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...