I'm writing a custom search command, and I'm running into the following error:
Failed to write buffer of size 21 to external process file descriptor (Broken pipe)
The custom search is an eventing command (command name is 'sum'):
import pandas as pd
import logging, logging.handlers
from splunklib.searchcommands import dispatch, EventingCommand, Configuration, Option, validators
def transform(self, records):
l = list(records)
l.sort(key=lambda r: r['_raw'])
if __name__ == "__main__":
dispatch(ExEventsCommand, sys.argv, sys.stdin, sys.stdout, __name__)
The error occurs only sometimes - it looks like it is dependent on the amount of data that is returned by the search.
This is illustrated by the following searches:
index = _internal | head 10000 | sum (no error)
index = _internal | head 100000 | sum (error)
In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B...
If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint.
View solution in original post