Splunk Enterprise

set-up hec behind load balancer

test_splunk15
Explorer

Hi,

we have a requirement where we need to send data from kinesis streams to Splunk via firehose using hec tokens.

In this use case I need to set-up infra in such a way that firehose should be configured with load balancer which is behind heavy forwarders (if one goes down other should be picked-up) -> which will then sends data to indexers.

As hec tokens are created individually on heavy forwarders, if we configure one hec on firehose and load balacer url there it will not be working, what is the best way to configure load balancer for firehose along with the tokens generated (we dont have cluster at HF).

referred below but they are not for firehose to HF load balancers 

https://docs.aws.amazon.com/firehose/latest/dev/writing-with-kinesis-streams.html

https://www.splunk.com/en_us/blog/cloud/power-data-ingestion-into-splunk-using-amazon-kinesis-data-f...

Let me know if you need more info.

 

Thanks

Labels (1)
Tags (1)
0 Karma

livehybrid
SplunkTrust
SplunkTrust

Hi @test_splunk15 ,

It sounds like you're generating unique HEC tokens on each indexer? If you want to be able to loadbalance your HEC then you will need the same set of tokens on each indexer.

If you're using a cluster then you can push these out using your cluster master (create an app in $SPLUNK_HOME/etc/master-apps containing the inputs.conf with your tokens.
If you're not using a cluster master then you could deploy out with a deployment server, and if not then you can manually copy the tokens between your indexers.

Once done you'll be able to loadbalance your HEC and the token will be accepted on any of the targets the LB hits.

I hope this helps, shout if you have any other questions

Thanks
Will 

Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...