Custom search command error: Failed to write buffer of size 21 to external process file descriptor

[BernardEAI]

Hi

I'm writing a custom search command, and I'm running into the following error:

Failed to write buffer of size 21 to external process file descriptor (Broken pipe)

The custom search is an eventing command (command name is 'sum'): 

#!/usr/bin/python
import exec_anaconda
exec_anaconda.exec_anaconda()
import pandas as pd
import os,sys
import logging, logging.handlers
import splunk

from splunklib.searchcommands import dispatch, EventingCommand, Configuration, Option, validators


@Configuration()
class ExEventsCommand(EventingCommand):
    def transform(self, records):
        l = list(records)
        l.sort(key=lambda r: r['_raw'])
        
        return l

if __name__ == "__main__":
    dispatch(ExEventsCommand, sys.argv, sys.stdin, sys.stdout, __name__)

The error occurs only sometimes - it looks like it is dependent on the amount of data that is returned by the search.

This is illustrated by the following searches:

index = _internal | head 10000 | sum (no error)
index = _internal | head 100000 | sum (error)

[thellmann · Accepted solution]

In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B...

If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint.