Getting Data In

How to configure the load balancer to handle HEC data?

danielbb
Motivator

We are in a transition from sending the data through HFs to sending the data directly to the indexers and we wonder how to configure the load balancer to handle this HTTP data. My understanding is that HTTP is based on TCP and TCP is connection based and therefore we can lock the sender to a particular indexer which would lead to an uneven distribution of the load, any suggestions?

Labels (1)
0 Karma

livehybrid
Super Champion

Hi @danielbb 

Are you running your infra on-premise or using a cloud service such as AWS? If you are using AWS Firehose to send data to HEC then there are specific requirements for loadbalancing (See https://docs.splunk.com/Documentation/AddOns/released/Firehose/ConfigureanELB)

Also, if you are using indexer acknowledgement with HEC then you need to ensure that (similar to Firehose sources) that your loadbalancer does cookie-based session stickiness so that the client can connect to the same indexer to check the acknowledgement.

Other than that, I believe any modern HTTP Load balancing product should work well.

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

0 Karma

danielbb
Motivator

Very interesting @livehybrid, how do I check whether indexer acknowledgment is in place?

0 Karma

livehybrid
Super Champion

You can check this on your existing inputs, if you have acknowledgement enabled you'll have the useAck set to true in your inputs.conf stanzas such as below:

[http://answers]
disabled = 0
host = macdev
index = answers
token = bbe67d25-6eca-41c3-9046-e1e9b75bb571
useAck = true

 

useACK = <boolean>
* When set to "true", acknowledgment (ACK) is enabled. Events in a request
  are tracked until they are indexed. An events status (indexed or not) can be
  queried from the ACK endpoint with the ID for the request.
* When set to false, acknowledgment is not enabled.
* This setting can be set at the stanza level.
* Default: false

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...