pipeline paralllelization ingest_pipe=0 not processing data

[aab5272]

Hi ,

I changed the pipelineparalleization=2

and i was able to see the data being processed through both pipelines, but after increasing the typingqueue , my one of ingest_pipe is not processing the data?

Can some one please let me know why would this happen. It was working before.

Thanks

[yannK]

You can use this search to look at your queues size per pipeline, and see where it gets full. ( we are using perc90 of the queue filled size)

  index=_internal host=* source=*metrics.log sourcetype=splunkd group=queue (name=parsingqueue OR name=aggqueue OR name=typingqueue OR name=indexqueue) | eval name=case(name=="aggqueue","2 - Aggregation Queue",name=="indexqueue","4 - Indexing Queue",name=="parsingqueue","1 - Parsing Queue",name=="typingqueue","3 - Typing Queue") | eval ingest_pipe = if(isnotnull(ingest_pipe), ingest_pipe, "none") | search ingest_pipe=* | eval max=if(isnotnull(max_size_kb),max_size_kb,max_size)  | eval curr=if(isnotnull(current_size_kb),current_size_kb,current_size)  | eval fill_perc=round((curr/max)*100,2) | eval name=name."-".ingest_pipe | timechart  Perc90(fill_perc) by name useother=false limit=15

We will order the processor in the order they run, so that way you can see if one is full and is backing up the previous ones.
1- parsing -> 2 -aggregation -> 3 -typing -> 4- indexing