I changed the pipelineparalleization=2
and i was able to see the data being processed through both pipelines, but after increasing the typingqueue , my one of ingest_pipe is not processing the data?
Can some one please let me know why would this happen. It was working before.
You can use this search to look at your queues size per pipeline, and see where it gets full. ( we are using perc90 of the queue filled size)
index=_internal host=* source=*metrics.log sourcetype=splunkd group=queue (name=parsingqueue OR name=aggqueue OR name=typingqueue OR name=indexqueue) | eval name=case(name=="aggqueue","2 - Aggregation Queue",name=="indexqueue","4 - Indexing Queue",name=="parsingqueue","1 - Parsing Queue",name=="typingqueue","3 - Typing Queue") | eval ingest_pipe = if(isnotnull(ingest_pipe), ingest_pipe, "none") | search ingest_pipe=* | eval max=if(isnotnull(max_size_kb),max_size_kb,max_size) | eval curr=if(isnotnull(current_size_kb),current_size_kb,current_size) | eval fill_perc=round((curr/max)*100,2) | eval name=name."-".ingest_pipe | timechart Perc90(fill_perc) by name useother=false limit=15
We will order the processor in the order they run, so that way you can see if one is full and is backing up the previous ones.
1- parsing -> 2 -aggregation -> 3 -typing -> 4- indexing