Splunk HF Parallel Pipelines not balanced

Utkc137 — Sun, 29 Oct 2023 15:39:04 GMT

We have configured parallelIngestionPipelines as 2 in Splunk HF as we were facing congestion in the TypingQueue while our CPU was underutilized (~2 Cores used/12).

However, the load in the pipelines are not balanced. Pipeline 0 is still congested while Pipeline 1 is barely utilized.

Digging around, this seems to be because 80% of our input is on a single UDP port.

Will splitting the UDP ports on the source itself solve this issue? i.e. having multiple UDP Inputs on the HF instead of one?

Re: Splunk HF Parallel Pipelines not balanced

isoutamo — Thu, 02 Nov 2023 13:55:52 GMT

much better option is use some real syslog server or SC4S to collect syslogs. And try to avoid use UDP as it always lost packets!

* If the data source is streamed over TCP or UDP, such as syslog sources, 
  only one pipeline will be used.

Based on that you cannot increase the UDP performance with adding pipelines.

r. Ismo

topic Splunk HF Parallel Pipelines not balanced in Monitoring Splunk

Splunk HF Parallel Pipelines not balanced

Re: Splunk HF Parallel Pipelines not balanced