Getting Data In

Setting up HEC (HTTP Event Collector) in a indexer cluster

patng_nw
Communicator

I am trying to set up HEC for my indexer cluster (v8.0.7), with 2 indexers (and 3 search heads) managed by a master node.

I read multiple docs and articles already, but I want to make sure I get some basic ideas correct first.

In a non-clustered env, it's simple and each HEC client will talk to port 8088 of one indexer. 

But in an indexer cluster environment:

  • Which server will an HEC client talk to? 
  • How can "load balancing of indexing" be achieved?
  • Is the master node in any way involved?
Labels (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

View solution in original post

0 Karma

patng_nw
Communicator

That's clear now.  I hope Splunk can update their doc to at least give us a high level picture.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@patng_nw  link Components that help to manage your deployment - Splunk Documentation having some info but not at HEC level. Set up and use HTTP Event Collector in Splunk Web - Splunk Documentation further reading.

Same you can use DS as well push to indexers, if you have HF in distributed set-up that's one of the place HEC can be enabled. It all depends how huge the load is on indexers.

0 Karma

patng_nw
Communicator

Thanks for the links.  I have read them before I post this question here, as they don't provide a clear high-level picture which list out the options we have.

 

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @patng_nw 

Following link having answers to your first two bullet points, go to very end of post How to deploy HEC and token to indexers in a clust... - Splunk Community

  • Configure HEC on master-node push to individual members, every cluster member acts as HEC receiver
  • You need to set-up a load balancer to front them to balance the load across they are not auto load balanced
  • HEC on master node is not advisable as it has a critical role to play do not overload with HEC traffic.

---

An upvote would be appreciated and Accept solution if this reply helps!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...