CPU Cores assigned to Index Pipeline

edoardo_vicendo — Wed, 20 Oct 2021 17:35:25 GMT

Hello,

In our environment we have Splunk HF with 2 parallel Ingestion Pipelines.

https://docs.splunk.com/Documentation/Splunk/8.2.2/Capacity/Parallelization#Index_parallelization

One of the aim of those Splunk HF is to offload the Splunk Indexer on parsing Pipeline, Merging Pipeline and Typing Pipeline. Due to that the data coming from Splunk HF are already "processed" and our Indexer are mostly processing them only in the Index Pipeline.

https://wiki.splunk.com/Community:HowIndexingWorks

On the Indexers we only have 1 Ingestion Pipeline, the CPU Cores used for indexing are typically 4-6.

Does our Indexers are taking advantage using pretty much all the 4-6 CPU Cores for the Index Pipeline only OR they are "wasted" on the other mostly idle pipelines?

Thanks a lot,
Edoardo

Re: CPU Cores assigned to Index Pipeline

isoutamo — Tue, 02 Nov 2021 22:00:26 GMT

How many source systems, HFs and indexers you have? Probably more interesting is how well your events are distributed over indexers than how well those cores/pipelines are used in any particular moment. Here is excellent tools to check this https://github.com/silkyrich/cluster_health_tools.

r. Ismo

topic CPU Cores assigned to Index Pipeline in Monitoring Splunk

CPU Cores assigned to Index Pipeline

Re: CPU Cores assigned to Index Pipeline