Splunk Search

External python search - How do I resolve Error "Failed to parse transport header"?

kepffr
Explorer

Hi guys,

I'm trying to write a very simple external python search but it's just not working.

I get the following error message in search_messages.log:

06-15-2021 09:44:22.543 +0200 ERROR SearchMessages - orig_component="script" app="search" sid="1623743052.198909" message_key="EXTERN:SCRIPT_NONZERO_RETURN" message=External search command 'pyTest' returned error code 1. Script output = "chunked 1.0,241,0\n{"inspector":{"messages":[["ERROR","RuntimeError at \"D:\\Splunk\\etc\\apps\\pyTest\\bin\\splunklib\\searchcommands\\search_command.py\", line 884 : Failed to parse transport header: b'splunkVersion:8.2.0\\n'"]]},"finished":true}".

 

It says message_key="EXTERN:SCRIPT_NONZERO_RETURN" and "Failed to parse transport header".

 This is how I call the script in a splunk search:

| makeresults 1 | eval something="just_a_value" | script pyTest

or

| script pyTest

 

This is my commands.conf:

[pyTest]
python.version = python3
chunked = true
filename = pyTest.py

 

This is my code:

#!/usr/bin/python3
import os, sys

sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "lib"))
import splunk.Intersplunk
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration

@Configuration()
class pyTest(StreamingCommand):
    def stream(self, events):
        for event in events:
            event['nothing'] = 'world'
            yield event

dispatch(pyTest, sys.argv, sys.stdin, sys.stdout, __name__)

 

I have also tried to replace \r\n with \n in the code but that didn't help. What am I doing wrong here?

Labels (1)
Tags (1)

kepffr
Explorer

Hi guys. Any ideas?

bpna
Explorer

I solved it! The solution reflects SO poorly on Splunk.

I ran my command with "--debug" and got this output:

Command list_entities appears to be statically configured for search command protocol version 1 and static configuration is unsupported by splunklib.searchcommands. Please ensure that default/commands.conf contains this stanza: [generatetext] filename = generatetext.py enableheader = true outputheader = true requires_srinfo = true supports_getinfo = true

If you add those settings to your commands.conf it should work

Splunk's OWN DOCS AND EXAMPLES IN MULTIPLE PLACES say "chunked = true" specifies search command protocol version 2

Like here https://dev.splunk.com/enterprise/docs/devtools/customsearchcommands/createcustomsearchcmd#Register-...

And here https://docs.splunk.com/Documentation/ITSI/4.12.0/Configure/commands.conf

And here https://github.com/splunk/splunk-sdk-python/blob/master/examples/searchcommands_app/package/default/...

Wow.

 

highsplunker
Contributor

Thank you! Worked OK!

I faced the same error. You helped a lot! 🙂

0 Karma

bpna
Explorer

I'm having the same issue.

Actually after pulling my hair out over it, copied the generatetext command example from Splunk SDK for Python GitHub repo into my instance https://github.com/splunk/splunk-sdk-python/blob/master/examples/searchcommands_app/package/bin/gene...

Copied default/commands.conf. Setup splunklib directory.

Splunk v8.2.2, should play nicely with python3

Still failing with this error. Did you ever solve it?

0 Karma

ohbuckeyeio
Path Finder

FWIW, I had this same problem and fixed it by changing my OS command from subprocess.run() to subprocess.Popen.  Not sure why, but it seem's Splunk's implementation of Python 3.7 has issues with subprocess.run() and streaming commands.

This allowed me to use 

chunked = true

without any problems. 

0 Karma

bpna
Explorer

That's interesting, thanks for the info. Helpful to know it's an issue with Splunk's usage of python internally.

I was able to solve it by editing commands.conf to the following:

[list_entities]
filename = list_entities.py
enableheader = true
outputheader = true
requires_srinfo = true
supports_getinfo = true
supports_multivalues = true
supports_rawargs = true
python.version = python3

 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...