Pipeline processors

[rroberts]

Running this search index=_internal "group=pipeline" | stats sum(cpu_seconds) as totalCPUSeconds by processor | sort 10 totalCPUSeconds desc
I see the sendout processor has the totalCPUseconds lead followed closely by indexer and aggregator processors. What exactly does the sendout processor do? I see it's a part of the "parsing"

[pipeline:parsing]

disabled_processors=utf8, linebreaker, header, sendOut

[jbsplunk · Accepted solution]

sendOut sends data from the parsingQueue to the aggQueue(aggregator). The parsing queue does UTF8, Linebreaker, header recognition, etc. aggQueue puts the events back together by using things like SHOULD_LINEMERGE, BREAK_ONLY_BEFORE, MUST_BREAK_AFTER, etc.

[krdo]

I know this a (very) old question/answer, but: I see that our Splunk indexer spends most cpu_seconds for the parsing sendout processor. What exactly does it do? The amount of cpu_seconds for all other processors is magnitudes lower. May this indicate a resource problem?

[jbsplunk]

yes. There is a send out between each queue, generally speaking.

[rroberts]

So is sendOut actually involved between each queue? IE parsingQueue to aggQueue to typingQueue and so on?