set-up hec behind load balancer

[test_splunk15]

Hi,

we have a requirement where we need to send data from kinesis streams to Splunk via firehose using hec tokens.

In this use case I need to set-up infra in such a way that firehose should be configured with load balancer which is behind heavy forwarders (if one goes down other should be picked-up) -> which will then sends data to indexers.

As hec tokens are created individually on heavy forwarders, if we configure one hec on firehose and load balacer url there it will not be working, what is the best way to configure load balancer for firehose along with the tokens generated (we dont have cluster at HF).

referred below but they are not for firehose to HF load balancers 

https://docs.aws.amazon.com/firehose/latest/dev/writing-with-kinesis-streams.html

https://www.splunk.com/en_us/blog/cloud/power-data-ingestion-into-splunk-using-amazon-kinesis-data-f...

Let me know if you need more info.

Thanks

[livehybrid]

Hi @test_splunk15 ,

It sounds like you're generating unique HEC tokens on each indexer? If you want to be able to loadbalance your HEC then you will need the same set of tokens on each indexer.

If you're using a cluster then you can push these out using your cluster master (create an app in $SPLUNK_HOME/etc/master-apps containing the inputs.conf with your tokens.
If you're not using a cluster master then you could deploy out with a deployment server, and if not then you can manually copy the tokens between your indexers.

Once done you'll be able to loadbalance your HEC and the token will be accepted on any of the targets the LB hits.

I hope this helps, shout if you have any other questions

Thanks
Will