Splunk Enterprise

pipeline paralllelization ingest_pipe=0 not processing data

aab5272
Engager

Hi ,

I changed the pipelineparalleization=2

and i was able to see the data being processed through both pipelines, but after increasing the typingqueue , my one of ingest_pipe is not processing the data?

Can some one please let me know why would this happen. It was working before.

Thanks

Tags (2)
0 Karma

yannK
Splunk Employee
Splunk Employee

You can use this search to look at your queues size per pipeline, and see where it gets full. ( we are using perc90 of the queue filled size)

  index=_internal host=* source=*metrics.log sourcetype=splunkd group=queue (name=parsingqueue OR name=aggqueue OR name=typingqueue OR name=indexqueue) | eval name=case(name=="aggqueue","2 - Aggregation Queue",name=="indexqueue","4 - Indexing Queue",name=="parsingqueue","1 - Parsing Queue",name=="typingqueue","3 - Typing Queue") | eval ingest_pipe = if(isnotnull(ingest_pipe), ingest_pipe, "none") | search ingest_pipe=* | eval max=if(isnotnull(max_size_kb),max_size_kb,max_size)  | eval curr=if(isnotnull(current_size_kb),current_size_kb,current_size)  | eval fill_perc=round((curr/max)*100,2) | eval name=name."-".ingest_pipe | timechart  Perc90(fill_perc) by name useother=false limit=15

We will order the processor in the order they run, so that way you can see if one is full and is backing up the previous ones.
1- parsing -> 2 -aggregation -> 3 -typing -> 4- indexing

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...