Getting Data In

Python 3 modular input on a universal forwarder version 8

jthunnissen
Path Finder

In light to the discontinuation of Python 2.7 we have upgraded both our universal forwarders (to version 😎 and our system python (to version 3.7). Unfortunately, the new UF does no longer seem to want to use the system python. In the logs it complains that ik cannot find C:\Program Files\SplunkForwarder\bin\Python3 which indeed does not exist, nor should it as this is a universal forwarder. Python 3 is in the PATH.

How do I get the UF to work with system python 3?

jbarlow_splunk
Splunk Employee
Splunk Employee

The fix is in 8.0.4 which is now available

May need to set python.version in server.conf (https://docs.splunk.com/Documentation/Splunk/latest/Admin/Serverconf)

python.version =
{python2|python3|force_python3}
* For Python scripts only, sets the default Python version to use.
* Can be overridden by other 'python.version' values elsewhere,
with the following exception:
* If you set to "force_python3", the system always uses Python 3, and
ignores 'python.version' values that
you set elsewhere.
* Default: python2

This will determine the name of the python executable it looks for when running python commands
(relevant python install needs to be in PATH)

If your instance uses python.exe,
python.version needs to be set to a value of:-
unspecified

This is not documented in current docs.

Set python.version accordingly :

python2 - python install uses python2/python2.exe
python3 - python install uses python3/python3.exe
unspecified - python install uses python/python.exe

Tested just now and it works.

jthunnissen
Path Finder

The following solution seemed to work at first. But if prevents Splunk from starting. Splunk support is currently still working on this issue for me.

server.conf:
[general]
python.version = force_python3

-my python 3.7.4 , which is in C:\Python3

-copy custom python.exe to C:\Program Files\SplunkUniversalForwarder\bin\Python3.exe

in C:\Program Files\SplunkUniversalForwarder\etc\splunk-launch.conf

PYTHONPATH=C:\Python3\lib
<<

  • restart splunk and custom python was picked up

splunk cmd python --version
Python 3.7.4
,I contacte Splunk support and the following solution worked for me:

server.conf:
[general]
python.version = force_python3

-my python 3.7.4 , which is in C:\Python3

-copy custom python.exe to C:\Program Files\SplunkUniversalForwarder\bin\Python3.exe

in C:\Program Files\SplunkUniversalForwarder\etc\splunk-launch.conf

PYTHONPATH=C:\Python3\lib
<<

  • restart splunk and custom python was picked up

splunk cmd python --version
Python 3.7.4

amrit
Splunk Employee
Splunk Employee

Hm, this sounds like a bug. I suppose in implementing Python versioning support, we inadvertently excluded system pythons by being very explicit about which Python interpreter we're going to execute.

No promises yet, but perhaps on the Universal Forwarder we can attempt to just execute whichever "python" is in $PATH...

0 Karma

andreaghetti
Engager

Hi Amrit,

any update o this? I'm affected by this issue too is there any workaround?

Thanks,
Andrea

amrit
Splunk Employee
Splunk Employee

This has been filed internally as SPL-183467. Please stay tuned.

0 Karma

BGarber42
Engager

Is there any progress? The 'accepted answer' prevents Splunk from even starting.

0 Karma

andreaghetti
Engager

The solution above worked for me. I tried it with both python2 and python3 and they worked.
To use Python2 I did not put the force_python3 value in server.conf
Are you sure those settings are the reason why your splunk universal forwarder is not starting?

0 Karma

BGarber42
Engager

Yes, when trying the workaround above it failed during a ./splunk start even with PYTHONPATH set to the system path as it was trying to load Python library files from $SPLUNK_HOME which didn't exist.

0 Karma

mbjerkeland_spl
Splunk Employee
Splunk Employee

This has been filed as a bug and is slated to be fixed in 8.0.4, according to SPL-183467.
Splunk UF should use the python in your PATH.

jthunnissen
Path Finder

Does is work for you when you use only python 3 (ergo force_python3) and then restart the UF? For me it fails fatally.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...