Deployment Architecture

Forwarders and Reverse Proxy

mzorzi
Splunk Employee
Splunk Employee

Currently, we've got four indexers and about 1400 forwarders. These forwarders all send their data to the indexers over the Internet.

The autoLB parameter is set on the forwarders to send to all four indexers. I'm curious if there is any way to set up a reverse proxy so that we can have one externally facing IP address and port, but then have the incoming traffic divided up between the indexers.

Many of our forwarders are at customer sites, and having them update their firewall rules every time we add a new indexer is frustrating for the customer and cumbersome for us.

dwaddle
SplunkTrust
SplunkTrust

Similar to what Ayn is suggesting I might do something like this:

  1. Set up a pair "bridgehead" light forwarders at each customer location (or require the customer to do so). Their local systems all forward to these.
  2. Also set up two (maybe more, always in pairs) heavy forwarders at your own location that are accessible by the customer bridgehead forwarders. The customer bridgeheads forward to your edge forwarders who can then offload parsing from your indexers and pass pre-parsed events on to your indexer farm.

This gives your customers a simple internal network / firewall configuration (at the cost of a couple of VMs), and gives you the ability to change your indexer footprint more or less at will.

Ayn
Legend

I should mention that this is precisely how we've set it up, and it seems we're running a similar service (architecture wise) to what is described in the initial question, and it works perfectly.

0 Karma

Ayn
Legend

If the intermediate forwarder is an Universal Forwarder, it has some default limits set, such as that it will send data at a maximum speed of 256kBps. That's just a default value though, that can easily be changed, as are all others. Heavy forwarders have no limits like this set by default. Either way, it's no bottleneck because it's essentially just another Splunk instance - just one that happens to forward stuff instead of indexing stuff.

andru
Explorer

If intermediate forwarders are used, do you know of any limitations? For example, how many forwarders could connect to a single intermediate forwarder before bottle necking would occur?

0 Karma

Ayn
Legend

Did you consider intermediate forwarders that use autoLB against backend indexers? If yes, what made you not choose that option?

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...