Splunk Enterprise

set-up hec behind load balancer

test_splunk15
Explorer

Hi,

we have a requirement where we need to send data from kinesis streams to Splunk via firehose using hec tokens.

In this use case I need to set-up infra in such a way that firehose should be configured with load balancer which is behind heavy forwarders (if one goes down other should be picked-up) -> which will then sends data to indexers.

As hec tokens are created individually on heavy forwarders, if we configure one hec on firehose and load balacer url there it will not be working, what is the best way to configure load balancer for firehose along with the tokens generated (we dont have cluster at HF).

referred below but they are not for firehose to HF load balancers 

https://docs.aws.amazon.com/firehose/latest/dev/writing-with-kinesis-streams.html

https://www.splunk.com/en_us/blog/cloud/power-data-ingestion-into-splunk-using-amazon-kinesis-data-f...

Let me know if you need more info.

 

Thanks

Labels (1)
Tags (1)
0 Karma

livehybrid
Builder

Hi @test_splunk15 ,

It sounds like you're generating unique HEC tokens on each indexer? If you want to be able to loadbalance your HEC then you will need the same set of tokens on each indexer.

If you're using a cluster then you can push these out using your cluster master (create an app in $SPLUNK_HOME/etc/master-apps containing the inputs.conf with your tokens.
If you're not using a cluster master then you could deploy out with a deployment server, and if not then you can manually copy the tokens between your indexers.

Once done you'll be able to loadbalance your HEC and the token will be accepted on any of the targets the LB hits.

I hope this helps, shout if you have any other questions

Thanks
Will 

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...