Splunk Search

External python search - How do I resolve Error "Failed to parse transport header"?

kepffr
Explorer

Hi guys,

I'm trying to write a very simple external python search but it's just not working.

I get the following error message in search_messages.log:

06-15-2021 09:44:22.543 +0200 ERROR SearchMessages - orig_component="script" app="search" sid="1623743052.198909" message_key="EXTERN:SCRIPT_NONZERO_RETURN" message=External search command 'pyTest' returned error code 1. Script output = "chunked 1.0,241,0\n{"inspector":{"messages":[["ERROR","RuntimeError at \"D:\\Splunk\\etc\\apps\\pyTest\\bin\\splunklib\\searchcommands\\search_command.py\", line 884 : Failed to parse transport header: b'splunkVersion:8.2.0\\n'"]]},"finished":true}".

 

It says message_key="EXTERN:SCRIPT_NONZERO_RETURN" and "Failed to parse transport header".

 This is how I call the script in a splunk search:

| makeresults 1 | eval something="just_a_value" | script pyTest

or

| script pyTest

 

This is my commands.conf:

[pyTest]
python.version = python3
chunked = true
filename = pyTest.py

 

This is my code:

#!/usr/bin/python3
import os, sys

sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "lib"))
import splunk.Intersplunk
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration

@Configuration()
class pyTest(StreamingCommand):
    def stream(self, events):
        for event in events:
            event['nothing'] = 'world'
            yield event

dispatch(pyTest, sys.argv, sys.stdin, sys.stdout, __name__)

 

I have also tried to replace \r\n with \n in the code but that didn't help. What am I doing wrong here?

Labels (1)
Tags (1)

kepffr
Explorer

Hi guys. Any ideas?

bpna
Explorer

I solved it! The solution reflects SO poorly on Splunk.

I ran my command with "--debug" and got this output:

Command list_entities appears to be statically configured for search command protocol version 1 and static configuration is unsupported by splunklib.searchcommands. Please ensure that default/commands.conf contains this stanza: [generatetext] filename = generatetext.py enableheader = true outputheader = true requires_srinfo = true supports_getinfo = true

If you add those settings to your commands.conf it should work

Splunk's OWN DOCS AND EXAMPLES IN MULTIPLE PLACES say "chunked = true" specifies search command protocol version 2

Like here https://dev.splunk.com/enterprise/docs/devtools/customsearchcommands/createcustomsearchcmd#Register-...

And here https://docs.splunk.com/Documentation/ITSI/4.12.0/Configure/commands.conf

And here https://github.com/splunk/splunk-sdk-python/blob/master/examples/searchcommands_app/package/default/...

Wow.

 

highsplunker
Contributor

Thank you! Worked OK!

I faced the same error. You helped a lot! 🙂

0 Karma

bpna
Explorer

I'm having the same issue.

Actually after pulling my hair out over it, copied the generatetext command example from Splunk SDK for Python GitHub repo into my instance https://github.com/splunk/splunk-sdk-python/blob/master/examples/searchcommands_app/package/bin/gene...

Copied default/commands.conf. Setup splunklib directory.

Splunk v8.2.2, should play nicely with python3

Still failing with this error. Did you ever solve it?

0 Karma

ohbuckeyeio
Path Finder

FWIW, I had this same problem and fixed it by changing my OS command from subprocess.run() to subprocess.Popen.  Not sure why, but it seem's Splunk's implementation of Python 3.7 has issues with subprocess.run() and streaming commands.

This allowed me to use 

chunked = true

without any problems. 

0 Karma

bpna
Explorer

That's interesting, thanks for the info. Helpful to know it's an issue with Splunk's usage of python internally.

I was able to solve it by editing commands.conf to the following:

[list_entities]
filename = list_entities.py
enableheader = true
outputheader = true
requires_srinfo = true
supports_getinfo = true
supports_multivalues = true
supports_rawargs = true
python.version = python3

 

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...