Custom search command error: Failed to write buffer of size 21 to external process file descriptor

BernardEAI — Mon, 30 Nov 2020 14:20:26 GMT

I'm writing a custom search command, and I'm running into the following error:

Failed to write buffer of size 21 to external process file descriptor (Broken pipe)

The custom search is an eventing command (command name is 'sum'):

#!/usr/bin/python import exec_anaconda exec_anaconda.exec_anaconda() import pandas as pd import os,sys import logging, logging.handlers import splunk from splunklib.searchcommands import dispatch, EventingCommand, Configuration, Option, validators @Configuration() class ExEventsCommand(EventingCommand): def transform(self, records): l = list(records) l.sort(key=lambda r: r['_raw']) return l if __name__ == "__main__": dispatch(ExEventsCommand, sys.argv, sys.stdin, sys.stdout, __name__)

The error occurs only sometimes - it looks like it is dependent on the amount of data that is returned by the search.

This is illustrated by the following searches:

index = _internal | head 10000 | sum (no error) index = _internal | head 100000 | sum (error)

Re: Custom search command error: Failed to write buffer of size 21 to external process file descriptor

thellmann — Tue, 08 Dec 2020 21:52:45 GMT

In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B.5D

If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint.

topic Custom search command error: Failed to write buffer of size 21 to external process file descriptor in Splunk Dev

Custom search command error: Failed to write buffer of size 21 to external process file descriptor

Re: Custom search command error: Failed to write buffer of size 21 to external process file descriptor