Splunk Dev

Custom search command error: Failed to write buffer of size 21 to external process file descriptor

BernardEAI
Communicator

Hi

I'm writing a custom search command, and I'm running into the following error:

Failed to write buffer of size 21 to external process file descriptor (Broken pipe)

The custom search is an eventing command (command name is 'sum'): 

 

#!/usr/bin/python
import exec_anaconda
exec_anaconda.exec_anaconda()
import pandas as pd
import os,sys
import logging, logging.handlers
import splunk

from splunklib.searchcommands import dispatch, EventingCommand, Configuration, Option, validators


@Configuration()
class ExEventsCommand(EventingCommand):
    def transform(self, records):
        l = list(records)
        l.sort(key=lambda r: r['_raw'])
        
        return l

if __name__ == "__main__":
    dispatch(ExEventsCommand, sys.argv, sys.stdin, sys.stdout, __name__)

 

The error occurs only sometimes - it looks like it is dependent on the amount of data that is returned by the search.

This is illustrated by the following searches:

 

index = _internal | head 10000 | sum (no error)
index = _internal | head 100000 | sum (error)

 

  

Labels (2)
0 Karma
1 Solution

thellmann
Splunk Employee
Splunk Employee

In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B...

If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint. 

View solution in original post

0 Karma

thellmann
Splunk Employee
Splunk Employee

In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B...

If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint. 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...