Splunk Enterprise

set-up hec behind load balancer

test_splunk15
Explorer

Hi,

we have a requirement where we need to send data from kinesis streams to Splunk via firehose using hec tokens.

In this use case I need to set-up infra in such a way that firehose should be configured with load balancer which is behind heavy forwarders (if one goes down other should be picked-up) -> which will then sends data to indexers.

As hec tokens are created individually on heavy forwarders, if we configure one hec on firehose and load balacer url there it will not be working, what is the best way to configure load balancer for firehose along with the tokens generated (we dont have cluster at HF).

referred below but they are not for firehose to HF load balancers 

https://docs.aws.amazon.com/firehose/latest/dev/writing-with-kinesis-streams.html

https://www.splunk.com/en_us/blog/cloud/power-data-ingestion-into-splunk-using-amazon-kinesis-data-f...

Let me know if you need more info.

 

Thanks

Labels (1)
Tags (1)
0 Karma

livehybrid
Builder

Hi @test_splunk15 ,

It sounds like you're generating unique HEC tokens on each indexer? If you want to be able to loadbalance your HEC then you will need the same set of tokens on each indexer.

If you're using a cluster then you can push these out using your cluster master (create an app in $SPLUNK_HOME/etc/master-apps containing the inputs.conf with your tokens.
If you're not using a cluster master then you could deploy out with a deployment server, and if not then you can manually copy the tokens between your indexers.

Once done you'll be able to loadbalance your HEC and the token will be accepted on any of the targets the LB hits.

I hope this helps, shout if you have any other questions

Thanks
Will 

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...