Heavy Forwarder queue saturation for sending logs to Universal Forwarders

[jeremieQuiviger]

Hello, 

I recently had to deploy a Heavy Forwarder in my infrastructure in order to perform transformations using a custom app. The current flow is as follows :

UF -> Heavy Forwarder relay -> Universal Forwarder relay -> Indexers

I am now observing a queue full issue on all Heavy Forwarders, while the queues on the next Universal Forwarders relay remain free.

I am therefore questioning the suitability of this architecture. I would like to know whether the Heavy Forwarder is still supposed to be at the end of the log flow, sending data directly to the Indexers, or if it can be positioned upstream of a Universal Forwarder that relays the data to the indexers.

Could this intermediate layer of Universal Forwarder be the cause of the queue saturation?

Thank you in advance for your help.

[gcusello]

Hi @jeremieQuiviger ,

why are you using another UF after the HF, you can use the HF for parsing and relay.

Anyway, did you checked the throughtput through the UFs? by default they have 256 kb, instead HF is unlimited.

The best solution is to replace the UF relay with an HF.

Ciao.

Giuseppe