Getting Data In

using a forwarder as a high-availability buffer?

tedder
Communicator

I have a couple of indexers behind a heavy forwarder, which reads from a batch of ports (and a few directories). If both of the indexers go down, incoming data is dropped on the floor. Is it possible to configure my forwarder to queue messages? It's bored, since it is just doing some basic parsing and throwing data towards the indexers. I can't be the first person to consider doing this.

1 Solution

gkanapathy
Splunk Employee
Splunk Employee

It is possible, but not recommended. You can review settings in outputs.conf to enable persistent queueing of data. Note that it's an especially bad idea if it's reading from files (vs transient network inputs), as the file content as well as the buffer will take up space on the machine. Also note that it's more likely for a single heavy forwarder to go down than for both a pair of indexers.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

It is possible, but not recommended. You can review settings in outputs.conf to enable persistent queueing of data. Note that it's an especially bad idea if it's reading from files (vs transient network inputs), as the file content as well as the buffer will take up space on the machine. Also note that it's more likely for a single heavy forwarder to go down than for both a pair of indexers.

gkanapathy
Splunk Employee
Splunk Employee

In most cases you really ought to have the indexes.conf settings configured so that indexers can't run out of space, and should discard the oldest data when it reaches the limits. This can be difficult if you have very limited storage however and lots of indexes.

0 Karma

tedder
Communicator

it's more likely that both indexers will run out of space- DAMHIK! Sort of a stampede issue- one runs out of space, so more traffic to the second causes issues. Thanks for the tips; knowing it is 'not recommended' is actually very helpful.

0 Karma
Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...