Splunk Dev

Custom search command error: Failed to write buffer of size 21 to external process file descriptor

BernardEAI
Communicator

Hi

I'm writing a custom search command, and I'm running into the following error:

Failed to write buffer of size 21 to external process file descriptor (Broken pipe)

The custom search is an eventing command (command name is 'sum'): 

 

#!/usr/bin/python
import exec_anaconda
exec_anaconda.exec_anaconda()
import pandas as pd
import os,sys
import logging, logging.handlers
import splunk

from splunklib.searchcommands import dispatch, EventingCommand, Configuration, Option, validators


@Configuration()
class ExEventsCommand(EventingCommand):
    def transform(self, records):
        l = list(records)
        l.sort(key=lambda r: r['_raw'])
        
        return l

if __name__ == "__main__":
    dispatch(ExEventsCommand, sys.argv, sys.stdin, sys.stdout, __name__)

 

The error occurs only sometimes - it looks like it is dependent on the amount of data that is returned by the search.

This is illustrated by the following searches:

 

index = _internal | head 10000 | sum (no error)
index = _internal | head 100000 | sum (error)

 

  

Labels (2)
0 Karma
1 Solution

thellmann
Splunk Employee
Splunk Employee

In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B...

If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint. 

View solution in original post

0 Karma

thellmann
Splunk Employee
Splunk Employee

In commands.conf, the default number of events that can be passed to a custom search command per invocation is determined by `maxinputs` and defaults to 50,000 - see here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Commandsconf#.5B.26lt.3BSTANZA_NAME.26gt.3B...

If your custom search command is expected to have input higher than that, you might investigate relaxing that constraint. 

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...