topic Re: External python search - Error "Failed to parse transport header" in Splunk Search

External python search - How do I resolve Error "Failed to parse transport header"?

kepffr — Tue, 02 Aug 2022 15:22:07 GMT

Hi guys,

I'm trying to write a very simple external python search but it's just not working.

I get the following error message in search_messages.log:

06-15-2021 09:44:22.543 +0200 ERROR SearchMessages - orig_component="script" app="search" sid="1623743052.198909" message_key="EXTERN:SCRIPT_NONZERO_RETURN" message=External search command 'pyTest' returned error code 1. Script output = "chunked 1.0,241,0\n{"inspector":{"messages":[["ERROR","RuntimeError at \"D:\\Splunk\\etc\\apps\\pyTest\\bin\\splunklib\\searchcommands\\search_command.py\", line 884 : Failed to parse transport header: b'splunkVersion:8.2.0\\n'"]]},"finished":true}".

It says message_key="EXTERN:SCRIPT_NONZERO_RETURN" and "Failed to parse transport header".

This is how I call the script in a splunk search:

| makeresults 1 | eval something="just_a_value" | script pyTest

| script pyTest

This is my commands.conf:

[pyTest] python.version = python3 chunked = true filename = pyTest.py

This is my code:

#!/usr/bin/python3 import os, sys sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "lib")) import splunk.Intersplunk from splunklib.searchcommands import dispatch, StreamingCommand, Configuration @Configuration() class pyTest(StreamingCommand): def stream(self, events): for event in events: event['nothing'] = 'world' yield event dispatch(pyTest, sys.argv, sys.stdin, sys.stdout, __name__)

I have also tried to replace \r\n with \n in the code but that didn't help. What am I doing wrong here?

Re: External python search - Error "Failed to parse transport header"

kepffr — Thu, 17 Jun 2021 07:26:47 GMT

Hi guys. Any ideas?

Re: External python search - Error "Failed to parse transport header"

bpna — Fri, 11 Mar 2022 00:15:48 GMT

I'm having the same issue.

Actually after pulling my hair out over it, copied the generatetext command example from Splunk SDK for Python GitHub repo into my instance https://github.com/splunk/splunk-sdk-python/blob/master/examples/searchcommands_app/package/bin/generatetext.py

Copied default/commands.conf. Setup splunklib directory.

Splunk v8.2.2, should play nicely with python3

Still failing with this error. Did you ever solve it?

Re: External python search - Error "Failed to parse transport header"

bpna — Fri, 11 Mar 2022 01:13:05 GMT

I solved it! The solution reflects SO poorly on Splunk.

I ran my command with "--debug" and got this output:

Command list_entities appears to be statically configured for search command protocol version 1 and static configuration is unsupported by splunklib.searchcommands. Please ensure that default/commands.conf contains this stanza: [generatetext] filename = generatetext.py enableheader = true outputheader = true requires_srinfo = true supports_getinfo = true

If you add those settings to your commands.conf it should work

Splunk's OWN DOCS AND EXAMPLES IN MULTIPLE PLACES say "chunked = true" specifies search command protocol version 2

Like here https://dev.splunk.com/enterprise/docs/devtools/customsearchcommands/createcustomsearchcmd#Register-your-custom-search-command

And here https://docs.splunk.com/Documentation/ITSI/4.12.0/Configure/commands.conf

And here https://github.com/splunk/splunk-sdk-python/blob/master/examples/searchcommands_app/package/default/commands.conf

Wow.

Re: External python search - Error "Failed to parse transport header"

ohbuckeyeio — Tue, 02 Aug 2022 15:21:44 GMT

FWIW, I had this same problem and fixed it by changing my OS command from subprocess.run() to subprocess.Popen. Not sure why, but it seem's Splunk's implementation of Python 3.7 has issues with subprocess.run() and streaming commands.

This allowed me to use

chunked = true

without any problems.

Re: External python search - Error "Failed to parse transport header"

bpna — Tue, 02 Aug 2022 16:02:52 GMT

That's interesting, thanks for the info. Helpful to know it's an issue with Splunk's usage of python internally.

I was able to solve it by editing commands.conf to the following:

[list_entities] filename = list_entities.py enableheader = true outputheader = true requires_srinfo = true supports_getinfo = true supports_multivalues = true supports_rawargs = true python.version = python3

Re: External python search - Error "Failed to parse transport header"

highsplunker — Wed, 19 Jul 2023 08:13:33 GMT

Thank you! Worked OK!

I faced the same error. You helped a lot! 🙂