Running this search index=_internal "group=pipeline" | stats sum(cpu_seconds) as totalCPUSeconds by processor | sort 10 totalCPUSeconds desc
I see the sendout processor has the totalCPUseconds lead followed closely by indexer and aggregator processors. What exactly does the sendout processor do? I see it's a part of the "parsing"
[pipeline:parsing]
disabled_processors=utf8, linebreaker, header, sendOut
sendOut sends data from the parsingQueue to the aggQueue(aggregator). The parsing queue does UTF8, Linebreaker, header recognition, etc. aggQueue puts the events back together by using things like SHOULD_LINEMERGE, BREAK_ONLY_BEFORE, MUST_BREAK_AFTER, etc.
sendOut sends data from the parsingQueue to the aggQueue(aggregator). The parsing queue does UTF8, Linebreaker, header recognition, etc. aggQueue puts the events back together by using things like SHOULD_LINEMERGE, BREAK_ONLY_BEFORE, MUST_BREAK_AFTER, etc.
I know this a (very) old question/answer, but: I see that our Splunk indexer spends most cpu_seconds for the parsing sendout processor. What exactly does it do? The amount of cpu_seconds for all other processors is magnitudes lower. May this indicate a resource problem?
yes. There is a send out between each queue, generally speaking.
So is sendOut actually involved between each queue? IE parsingQueue to aggQueue to typingQueue and so on?