topic Re: Python 3 modular input on a universal forwarder version 8 in Getting Data In

Python 3 modular input on a universal forwarder version 8

jthunnissen — Tue, 21 Jan 2020 17:06:05 GMT

In light to the discontinuation of Python 2.7 we have upgraded both our universal forwarders (to version 😎 and our system python (to version 3.7). Unfortunately, the new UF does no longer seem to want to use the system python. In the logs it complains that ik cannot find C:\Program Files\SplunkForwarder\bin\Python3 which indeed does not exist, nor should it as this is a universal forwarder. Python 3 is in the PATH.

How do I get the UF to work with system python 3?

Re: Python 3 modular input on a universal forwarder version 8

amrit — Wed, 12 Feb 2020 00:57:00 GMT

Hm, this sounds like a bug. I suppose in implementing Python versioning support, we inadvertently excluded system pythons by being very explicit about which Python interpreter we're going to execute.

No promises yet, but perhaps on the Universal Forwarder we can attempt to just execute whichever "python" is in $PATH...

Re: Python 3 modular input on a universal forwarder version 8

andreaghetti — Wed, 04 Mar 2020 14:46:52 GMT

Hi Amrit,

any update o this? I'm affected by this issue too is there any workaround?

Thanks,
Andrea

Re: Python 3 modular input on a universal forwarder version 8

jthunnissen — Thu, 05 Mar 2020 12:27:22 GMT

The following solution seemed to work at first. But if prevents Splunk from starting. Splunk support is currently still working on this issue for me.

server.conf:
[general]
python.version = force_python3

-my python 3.7.4 , which is in C:\Python3

-copy custom python.exe to C:\Program Files\SplunkUniversalForwarder\bin\Python3.exe

in C:\Program Files\SplunkUniversalForwarder\etc\splunk-launch.conf

PYTHONPATH=C:\Python3\lib
<<

restart splunk and custom python was picked up

splunk cmd python --version
Python 3.7.4
,I contacte Splunk support and the following solution worked for me:

server.conf:
[general]
python.version = force_python3

-my python 3.7.4 , which is in C:\Python3

-copy custom python.exe to C:\Program Files\SplunkUniversalForwarder\bin\Python3.exe

in C:\Program Files\SplunkUniversalForwarder\etc\splunk-launch.conf

PYTHONPATH=C:\Python3\lib
<<

restart splunk and custom python was picked up

splunk cmd python --version
Python 3.7.4

Re: Python 3 modular input on a universal forwarder version 8

amrit — Thu, 05 Mar 2020 23:55:08 GMT

This has been filed internally as SPL-183467. Please stay tuned.

Re: Python 3 modular input on a universal forwarder version 8

BGarber42 — Wed, 15 Apr 2020 15:33:59 GMT

Is there any progress? The 'accepted answer' prevents Splunk from even starting.

Re: Python 3 modular input on a universal forwarder version 8

andreaghetti — Thu, 16 Apr 2020 07:52:51 GMT

The solution above worked for me. I tried it with both python2 and python3 and they worked.
To use Python2 I did not put the force_python3 value in server.conf
Are you sure those settings are the reason why your splunk universal forwarder is not starting?

Re: Python 3 modular input on a universal forwarder version 8

jthunnissen — Thu, 16 Apr 2020 08:19:29 GMT

Does is work for you when you use only python 3 (ergo force_python3) and then restart the UF? For me it fails fatally.

Re: Python 3 modular input on a universal forwarder version 8

BGarber42 — Thu, 16 Apr 2020 16:44:07 GMT

Yes, when trying the workaround above it failed during a ./splunk start even with PYTHONPATH set to the system path as it was trying to load Python library files from $SPLUNK_HOME which didn't exist.

Re: Python 3 modular input on a universal forwarder version 8

mbjerkeland_spl — Tue, 28 Apr 2020 06:55:34 GMT

This has been filed as a bug and is slated to be fixed in 8.0.4, according to SPL-183467.
Splunk UF should use the python in your PATH.

Re: Python 3 modular input on a universal forwarder version 8

jbarlow_splunk — Wed, 30 Sep 2020 05:29:52 GMT

The fix is in 8.0.4 which is now available

May need to set python.version in server.conf (https://docs.splunk.com/Documentation/Splunk/latest/Admin/Serverconf)

python.version =
{python2|python3|force_python3}
* For Python scripts only, sets the default Python version to use.
* Can be overridden by other 'python.version' values elsewhere,
with the following exception:
* If you set to "force_python3", the system always uses Python 3, and
ignores 'python.version' values that
you set elsewhere.
* Default: python2

This will determine the name of the python executable it looks for when running python commands
(relevant python install needs to be in PATH)

If your instance uses python.exe,
python.version needs to be set to a value of:-
unspecified

This is not documented in current docs.

Set python.version accordingly :

python2 - python install uses python2/python2.exe
python3 - python install uses python3/python3.exe
unspecified - python install uses python/python.exe

Tested just now and it works.