topic Re: Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7? in Splunk Dev

Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7?

sjalexander — Fri, 13 Nov 2015 21:17:40 GMT

This isn't a question, but a heads-up.

On Red Hat 7, the distro Python and the Splunk Python don't get along. Red Hat's management tools are written in Python so this can be especially problematic.

This can become an issue if you load /opt/splunk/bin/setSplunkEnv into your environment - the $PATH variable, among others, is modified so that Splunk's Python takes precedence; Red Hat's tools or anything else callingPython will try to use Splunk's python.

Most operations don't throw stack traces, so "yum update" for example gave me nothing to go on. But I got the following stack trace on running rhn-profile-sync with the Splunk environment loaded:

backtrace:
:__init__.py:36:<module>:ImportError: /opt/splunk/lib/python2.7/site-packages/OpenSSL/crypto.so: undefined symbol: PyUnicodeUCS2_Decode
:
:Traceback (most recent call last):
:  File "/usr/sbin/rhn-profile-sync", line 20, in <module>
:    from up2date_client import up2dateAuth
:  File "/usr/share/rhn/up2date_client/up2dateAuth.py", line 4, in <module>
:    import rhnserver
:  File "/usr/share/rhn/up2date_client/rhnserver.py", line 34, in <module>
:    import rpcServer
:  File "/usr/share/rhn/up2date_client/rpcServer.py", line 6, in <module>
:    import config
:  File "/usr/share/rhn/up2date_client/config.py", line 18, in <module>
:    from rhn.connections import idn_ascii_to_pune, idn_pune_to_unicode
:  File "/usr/lib/python2.7/site-packages/rhn/connections.py", line 12, in <module>
:    import SSL
:  File "/usr/lib/python2.7/site-packages/rhn/SSL.py", line 27, in <module>
:    from OpenSSL import SSL
:  File "/opt/splunk/lib/python2.7/site-packages/OpenSSL/__init__.py", line 36, in <module>
:    from OpenSSL import crypto
:ImportError: /opt/splunk/lib/python2.7/site-packages/OpenSSL/crypto.so: undefined symbol: PyUnicodeUCS2_Decode

That's how I found the error - the stack trace points to the Splunk Python.

The undefined symbol: PyUnicodeUCS2_Decode issue is detailed here: https://docs.python.org/2/faq/extending.html#when-importing-module-x-why-do-i-get-undefined-symbol-pyunicodeucs2

In a nutshell, Red Hat's Python is built differently from Splunk's Python, so make sure you use Splunk's Python for Splunk operations, and Red Hat's Python for Red Hat operations, and you should be good to go.

Re: Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7?

ppablo — Fri, 13 Nov 2015 21:26:41 GMT

Hey there @sjalexander

Thanks for sharing this. Would you actually be able to post your "Answer" explaining why this error occurs in the "Enter your answer here..." box below and officially accept? Otherwise, this will float around in limbo as unresolved with no answer.

Re: Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7?

sjalexander — Fri, 13 Nov 2015 21:27:18 GMT

yeah can do.

Re: Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7?

sjalexander — Fri, 13 Nov 2015 21:28:18 GMT

In a nutshell, Red Hat's python is built differently from splunk's python.

Make sure you use splunk's python for splunk operations, and Red Hat's python for Red Hat operations, and you should be good to go.

In other words, if you're going to do Red Hat admin work, don't load /opt/splunk/bin/setSplunkEnv

Re: Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7?

sjalexander — Fri, 13 Nov 2015 21:29:27 GMT

@ppablo since you're here - is there a cleaner way of formatting code blocks? I gave up after trying maybe 5 times. The preview and the actual output don't match.

Re: Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7?

ppablo — Fri, 13 Nov 2015 21:50:08 GMT

Ah yeah, so I actually did fix the formatting for you already, but I think you were editing at the same time, so your recent save overwrote my changes lol. I'll go ahead and fix it up for you.

For any sample code, error messages, searches, and conf file stanzas (especially ones with regular expressions), it's best to highlight the entire block of that content and click the Code Sample button in the text editing tools like so:

[stanza...]
REGEX = \*blahregex*s\\*\*

If you're trying to render special characters properly inline within a sentence/paragraph, you just need to surround it with backticks like this: \*blahregex*s\\*\*

The only characters that don't display properly within backticks are < and > and a bug is filed for that.

Hope that helps!

Re: Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7?

sjalexander — Fri, 13 Nov 2015 23:06:02 GMT

It helps a lot. Thanks!

Re: Why am I getting "ImportError: undefined symbol: PyUnicodeUCS2_Decode" with Splunk on Red Hat Linux 7?

ppablo — Tue, 17 Nov 2015 02:14:42 GMT

Awesome, no problem. And thanks for sharing this post/tip with the rest of the Splunk community. Cheers!

Patrick