Developing for Splunk Enterprise

Custom search command error: Failed to write buffer of size 21 to external process file descriptor

BernardEAI
Communicator

Hi

I'm writing a custom search command, and I'm running into the following error:

Failed to write buffer of size 21 to external process file descriptor (Broken pipe)

The custom search is an eventing command (command name is 'sum'): 

 

#!/usr/bin/python
import exec_anaconda
exec_anaconda.exec_anaconda()
import pandas as pd
import os,sys
import logging, logging.handlers
import splunk

from splunklib.searchcommands import dispatch, EventingCommand, Configuration, Option, validators


@Configuration()
class ExEventsCommand(EventingCommand):
    def transform(self, records):
        l = list(records)
        l.sort(key=lambda r: r['_raw'])
        
        return l

if __name__ == "__main__":
    dispatch(ExEventsCommand, sys.argv, sys.stdin, sys.stdout, __name__)

 

The error occurs only sometimes - it looks like it is dependent on the amount of data that is returned by the search.

This is illustrated by the following searches:

 

index = _internal | head 10000 | sum (no error)
index = _internal | head 100000 | sum (error)

 

  

Labels (2)
0 Karma
1 Solution

thellmann
Splunk Employee
Splunk Employee

In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B...

If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint. 

View solution in original post

0 Karma

thellmann
Splunk Employee
Splunk Employee

In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B...

If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint. 

View solution in original post

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.