heavy forwarder multi core?

[danan5]

Hi,

I see a few similar questions in the past however I can't see an answer to date.

Does anyone know if the splunkd process can utilise modern multicore cpu architectures? or is it bound to a single thread of execution?

The splunk ingest pipeline looks to me (as a non-programmer) as something that might lend itself to multi-threading.

Thanks,
David

[codebuilder · Accepted solution]

If you increase the parallel ingestion pipelines on the forwarder (default = 1), it will utilize one core per pipeline.

https://docs.splunk.com/Documentation/Splunk/8.0.1/Indexer/Pipelinesets#Forwarders_and_multiple_pipe...

One caveat worth noting, when handling compressed files (.gz) the forwarder will utilize a single pipeline only, regardless of the pipeline setting.

[adonio]

see here:
https://answers.splunk.com/answers/12027/single-multi-threading-cpu.html

maybe this will help as well:
https://docs.splunk.com/Documentation/Splunk/8.0.1/Capacity/Parallelization