Getting Data In

Setting up HEC (HTTP Event Collector) in a indexer cluster

patng_nw
Communicator

I am trying to set up HEC for my indexer cluster (v8.0.7), with 2 indexers (and 3 search heads) managed by a master node.

I read multiple docs and articles already, but I want to make sure I get some basic ideas correct first.

In a non-clustered env, it's simple and each HEC client will talk to port 8088 of one indexer. 

But in an indexer cluster environment:

  • Which server will an HEC client talk to? 
  • How can "load balancing of indexing" be achieved?
  • Is the master node in any way involved?
Labels (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

View solution in original post

patng_nw
Communicator

That's clear now.  I hope Splunk can update their doc to at least give us a high level picture.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@patng_nw  link Components that help to manage your deployment - Splunk Documentation having some info but not at HEC level. Set up and use HTTP Event Collector in Splunk Web - Splunk Documentation further reading.

Same you can use DS as well push to indexers, if you have HF in distributed set-up that's one of the place HEC can be enabled. It all depends how huge the load is on indexers.

0 Karma

patng_nw
Communicator

Thanks for the links.  I have read them before I post this question here, as they don't provide a clear high-level picture which list out the options we have.

 

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...