Getting Data In

Indexer Acknowledgement and stalling of data flow

tomrit
Explorer

Hi!

I have a setup where I must clone and forward data to a third party. Can somebody clarify if I disable useACK that even though a destination is unreachable that the flow to other outputs does not stall? If I look at the spec of outputs.conf, for me it's not fully clear what happens when I write to the socket and the destination is unreachable:

 

* When set to "false", the forwarder considers the data fully processed
  when it finishes writing it to the network socket.

 

Thanks in advance!

Labels (2)
0 Karma
1 Solution

acharlieh
Influencer

Even with useAck disabled, if one side is completely blocked, then Splunk cannot send so the output Queue could back up and can eventually cause sending to all outputs to halt.

(on a forwarder with useAck true the forwarder  removes the data from its buffer after it gets acknowledgement that the indexer finished processing the data. With useAck false, the forwarder removes the data from the buffer after the indexer successfully received the data… if the forwarder cannot connect in either case, it fills the buffer)

I think you’re looking for something like setting

dropEventsOnQueueFull

On the third party output, others may have more experience with tuning this and similar settings ref: 

https://docs.splunk.com/Documentation/Splunk/8.2.3/Admin/Outputsconf

 

View solution in original post

tomrit
Explorer

Thanks a lot for the fast clarification!

0 Karma

acharlieh
Influencer

Even with useAck disabled, if one side is completely blocked, then Splunk cannot send so the output Queue could back up and can eventually cause sending to all outputs to halt.

(on a forwarder with useAck true the forwarder  removes the data from its buffer after it gets acknowledgement that the indexer finished processing the data. With useAck false, the forwarder removes the data from the buffer after the indexer successfully received the data… if the forwarder cannot connect in either case, it fills the buffer)

I think you’re looking for something like setting

dropEventsOnQueueFull

On the third party output, others may have more experience with tuning this and similar settings ref: 

https://docs.splunk.com/Documentation/Splunk/8.2.3/Admin/Outputsconf

 

Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...