Getting Data In

Indexer goes down, Stop the Splunk-forwarder from sending

Dark_Ichigo
Builder

I have been doing some research into what to my Splunk Forwarder can happen if my Splunk indexer goes down, I have noticed that it may will continue to queue the data in memory until full, this isn't what I had in mind as it will fill up what I can send through the bandwidth connection between the forwarder and the indexer.

I have noticed that there is a useACK=true in outputs.conf on the forwarder side, ifset to true I can then set both readTimeout & writeTimeot parameters to stop when the is no acknowledgement from the Splunk indexer, I have no idea if indeed setting the acknowledgement functionality also includes me setting up how much i want to indexer to index before sending in another patch for indexing, does this have to be specified manually?, is the useACK=true the best option I have?

As I would rather have the Splunk Forwarder stop sending through anything once the Splunk indexer indeed goes down.

0 Karma

yannK
Splunk Employee
Splunk Employee

This is already the default behavior:
if the indexer refuses data (full or down) then the forwarders fill their memory queue up to 2MB (default), then pause the monitoring.

Using useACK will only be useful if you have an unreliable connection and want the indexers to send an acknowledgment to the forwarders who otherwise will resent the events.

martin_mueller
SplunkTrust
SplunkTrust
0 Karma

Dark_Ichigo
Builder

Thanks, but I take it we need to set useACK=true first?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

In outputs.conf:

[tcpout]
maxQueueSize = 1MB
0 Karma

Dark_Ichigo
Builder

Is there a way to modify the max queue in the case to higher or lower than the default 2MB to stop it from forwarding data?

Does it keep logging events in the logs after it stops as well, repeatedly that its failing to send?

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...