Python script pulls data fine outside of Splunk Enterprise

[agw]

Hello-

When I run "splunk cmd python scripts\test.py" it outputs data nicely.  When I setup this through Splunk Enterprise Web, it errors out with:

File "C:\Program Files\Splunk\bin\scripts\TEST.py", line 58

except HTTPError, e:

                                     ^

SyntaxError: invalid syntax

I tried this from a different machine that had Python installed and from the windows command prompt it outputs the data fine.  Why doesn't this work in Splunk Enterprise?

[jhanvidattani]

This except HTTPError, e: syntax works only for python2. For code to be python2 and python3 compatible, the syntax would be: except HTTPError as e:

You can change the syntax as mentioned above, for code to be both python2 and python3 compatible. Also, by doing this you won't need to hardcode the python version.

Can you check the python version for the machine in which it is successfully executing? I feel it would be py2 and not py3. 

[jhanvidattani]

@agw 

This except HTTPError, e: syntax works only for python2. FOr code to be python2 and python3 compatible, the syntax would be: except HTTPError as e:

Can you check the python version with which the script is being executed? It seems that the code is compatible with python2 and the script is running with python3.

If you find my solution fruitful, an upvote would be appreciated.

[agw · Accepted solution]

So this is working now and I figured out the problem.  The script source type was set to "_json".  Once I changed that to "syslog" the data started pulling correctly without error.  The information I have read on API data, it should be set to _json but whatever works I guess.

[jhanvidattani]

@agw 

Happy to hear that the issue is resolved 🙂

But I believe that the syntax error for except statement is due to the different python version. Change in the value of source would solve below error and not error for except statement:

JsonLineBreaker - JSON StreamId:17225535322107805233 had parsing error:Unexpected character while expecting '"': '\'' - data_source="C:\Program Files\Splunk\bin\scripts\NEW.py", data_host="splunkindexer1", data_sourcetype="_json"

Correct me if am missing anything.

[agw]

So the except statement was corrected by editing the script, yes but then the script would not work in Splunk Enterpise but would work on another machine running python3.  It never worked in Splunk Enterprise until I changed from json to syslog.  Also it confused me that some of the script versions worked in Python2 and Python3 but not in Splunk Enterprise.  I really appreciate your help though and your questions helped me get to the solution. 

[agw]

On the other server when I run python -v I get Python 3.9.6.  I have gone through and modified the script to work in Python3 as a test but it's failing in Splunk Enterprise as well.  I will post the errors I'm getting.

[agw]

So the error that shows on the working Python3 code on Splunk Enterprise now is:

JsonLineBreaker - JSON StreamId:17225535322107805233 had parsing error:Unexpected character while expecting '"': '\'' - data_source="C:\Program Files\Splunk\bin\scripts\NEW.py", data_host="splunkindexer1", data_sourcetype="_json"

[agw]

Yes I know the command can fail on various versions of python.  The problem is the command  "splunk cmd python scripts\test.py" is being run directly on the Splunk server which outputs the result fine. 

[jhanvidattani]

@agw 

For this can you execute the below command from the bin directory to check the default version of python used by Splunk?

./splunk cmd python -V

[agw]

So I ran the command and it shows Python 2.7.17.  However both python 2.7 and 3.7 show up in the Splunk directory on the server. Is there a way to force version 2.7 on a script in Splunk Enterprise?

[jhanvidattani]

Yes, there is a way for this. You can set the value of the python version as mentioned here: Link 

[agw]

So my Splunk enterprise is already set to version 3 and I'm using Splunk Enterprise version 8.0.5.  As I said in the original post the python script on a different machine works fine and it is using only version 3 on that server, and also from the command line directly on the Splunk server it is using 2.7.  Changing the enterprise config to force python version 2 on the splunk server could break all kinds of other things that I don't want to break.  And why would I need to change that when the script clearly works in both python 2.7 and python 3.  It just doesn't work on the Splunk Enterprise website.  So I'm at a loss on what to do.