topic Re: Pipeline processors in Splunk Enterprise

Pipeline processors

rroberts — Mon, 28 Sep 2020 11:46:49 GMT

Running this search index=_internal "group=pipeline" | stats sum(cpu_seconds) as totalCPUSeconds by processor | sort 10 totalCPUSeconds desc
I see the sendout processor has the totalCPUseconds lead followed closely by indexer and aggregator processors. What exactly does the sendout processor do? I see it's a part of the "parsing"

[pipeline:parsing]

disabled_processors=utf8, linebreaker, header, sendOut

Re: Pipeline processors

jbsplunk — Mon, 28 Sep 2020 11:46:51 GMT

sendOut sends data from the parsingQueue to the aggQueue(aggregator). The parsing queue does UTF8, Linebreaker, header recognition, etc. aggQueue puts the events back together by using things like SHOULD_LINEMERGE, BREAK_ONLY_BEFORE, MUST_BREAK_AFTER, etc.

Re: Pipeline processors

rroberts — Mon, 07 May 2012 22:04:58 GMT

So is sendOut actually involved between each queue? IE parsingQueue to aggQueue to typingQueue and so on?

Re: Pipeline processors

jbsplunk — Mon, 07 May 2012 22:12:41 GMT

yes. There is a send out between each queue, generally speaking.

Re: Pipeline processors

krdo — Tue, 29 Sep 2020 23:30:04 GMT

I know this a (very) old question/answer, but: I see that our Splunk indexer spends most cpu_seconds for the parsing sendout processor. What exactly does it do? The amount of cpu_seconds for all other processors is magnitudes lower. May this indicate a resource problem?