topic universal forwarder in Getting Data In

universal forwarder

hazem — Tue, 04 Mar 2025 14:38:03 GMT

Dear all,

I have the following outputs.conf configuration:

[tcpout] defaultGroup = my_indexers

[tcpout:my_indexers] server = mysplunk_indexer1:9997, mysplunk_indexer2:9997

[tcpout-server://mysplunk_indexer1:9997]

Could you please clarify the Universal Forwarder (UF) behavior in the event that mysplunk_indexer1 goes down?

Will the UF continue sending data to both indexers despite mysplunk_indexer1 being down?
Or will the UF detect that mysplunk_indexer1 is unreachable and stop forwarding traffic to it?

Re: universal forwarder

livehybrid — Tue, 04 Mar 2025 14:42:36 GMT

Hi @hazem

Will the UF continue sending data to both indexers?
No, it will only send data to the available indexer (mysplunk_indexer2)

Will the UF detect that mysplunk_indexer1 is unreachable?
Yes, the UF will detect the unreachability and automatically adjust its forwarding strategy

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

Re: universal forwarder

livehybrid — Tue, 04 Mar 2025 14:43:34 GMT

In terms of further breakdown to the previous answer:

Automatic Failover: If mysplunk_indexer1 goes down, the UF will detect the failure and automatically stop sending data to that indexer.
Continued Forwarding to Available Indexers: The UF will continue forwarding data to mysplunk_indexer2:9997. The forwarder does not stop forwarding entirely but rather distributes the load among the remaining available indexers.
Retry Logic: The UF will periodically attempt to reconnect to mysplunk_indexer1. Once it becomes available again, data will resume being sent to it.
Load Balancing (if applicable): If both indexers were previously receiving traffic in a load-balanced manner (e.g., using autoLBFrequency), the UF would shift all the load to the remaining functional indexer.

Also, you might want to consider the following:

If no indexers are available, events will be queued locally in memory (or on disk if useAck is enabled).
Ensure you configure proper connectionTimeout and autoLBFrequency settings to optimize failover behavior.
If useACK=true (for reliable delivery), the UF will queue events until an indexer acknowledges them.

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

Re: universal forwarder

gcusello — Tue, 04 Mar 2025 15:35:38 GMT

Hi @hazem ,

at first the last row isn't mandatory, it's an old configuration and if you put it, you should add one row for each server.

Anyway, if you configure more than one Indexer, lofs are forwarded to all the Indexers changing destination every 30 seconds using a round robin algorithm for the load balancing.

Then, if an Indexers isn't available, the Forwarders tries with another one; id no Indexers are available it saves logs on a local cache and forward them when the connection is established again.

Ciao.

Giuseppe