Which resources are affecting Splunk Enterprise Data Pipeline?

[rbakeredfi]

When the index pipeline begins backing up at any stage, which resources are responsible for the bottleneck. Obviously, once backed up the problem will overflow into other areas but is there a "rule" or anything that says if the backup is at the Parsing Pipeline then the storage IO is too low,  Merging Pipeline then the CPU is too low,  Typing Pipeline the memory is too low, or Index Pipeline it's network bandwidth, etc. I am specifically looking for info regarding a Heavy Forwarder but any help would be appreciated.

*It's not as bad as the picture makes it seem, just posting for visual*

[gcusello]

Hi @rbakeredfi,

are you speaking of an Indexer or an Heavy Forwarder?

Have you done a correct assignemt of resources? how many CPUs have you on this server?

If you're speaking of an Indexer, have you a performant disk: at least 800 IOPS (better 1200)?

Ciao.

Giuseppe

[rbakeredfi]

I am looking for more of a generic mapping of resources to parts of the pipeline.

However, this specific case is regarding a HF.

Machine Name	Machine CPU Cores (Physical / Virtual)	Physical Memory Capacity (MB)	Operating System	Architecture
redacted	16 / 32	131020	Windows	x64

[gcusello]

Hi @rbakeredfi,

forgetting for a moment the use of Windows that I'd avoid in production systems!

how many logs this HF must manage?

are there many syslogs? if yes how do you input them using Splunk inputs or an external rsyslog server?

Are you sure to have a performant network between the HF and the Indexers?

are Indexers overloaded or not?

Ciao.

Giuseppe