Why am I getting "ERROR:root:code for hash md5 was...

jodros · ‎04-15-2016

I am having an issue running a python script from my dev Splunk search head. I keep getting this message:

ERROR:root:code for hash md5 was not found.

I am running this on RHEL 6.7. I have checked and openssl is installed and up to date. Python2.6 is also pre-installed with RHEL6.7. However with I run the python script I am using Python2.7 which is installed with Splunk 6.3. Some diagnostic outputs are below.

[johndr@HO-SPLUNKES ~]$ which python
/opt/splunk/bin/python
[johndr@HO-SPLUNKES ~]$ python
Python 2.7.9 (default, Jul 15 2015, 01:30:03)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> help("modules")

Please wait a moment while I gather a list of all available modules...

ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/opt/splunk/lib/python2.7/hashlib.py", line 97, in __get_builtin_constru                                                                                                                                                             ctor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/opt/splunk/lib/python2.7/hashlib.py", line 97, in __get_builtin_constru                                                                                                                                                             ctor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha1
BaseHTTPServer      bz2                 io                  sets
Bastion             cPickle             itertools           sgmllib
CDROM               cProfile            json                sha
CGIHTTPServer       cStringIO           keyword             shelve
ConfigParser        calendar            linecache           shlex
Cookie              cgi                 locale              shutil
DLFCN               cgitb               logging             signal
DocXMLRPCServer     cherrypy            lxml                site
HTMLParser          chunk               macpath             sitecustomize
IN                  cmd                 macurl2path         smtpd
IPy                 code                mailbox             smtplib
M2Crypto            codecs              mailcap             sndhdr
MimeWriter          codeop              mako                socket
OpenSSL             collections         markupbase          splunk
Queue               colorsys            marshal             sre
SimpleHTTPServer    commands            math                sre_compile
SimpleXMLRPCServer  compileall          md5                 sre_constants
SocketServer        compiler            mhlib               sre_parse
StringIO            contextlib          mimetools           ssl
TYPES               cookielib           mimetypes           stat
UserDict            copy                mimify              statvfs
UserList            copy_reg            modulefinder        string
UserString          cssmin              multifile           stringold
_LWPCookieJar       csv                 multiprocessing     stringprep
_MozillaCookieJar   datetime            mutex               strop
__builtin__         dbhash              netrc               struct
__future__          decimal             new                 subprocess
_abcoll             decorator           nntplib             sunau
_ast                difflib             ntpath              sunaudio
_bisect             dircache            nturl2path          symbol
_codecs             dis                 numbers             symtable
_collections        distutils           opcode              sys
_csv                django              operator            sysconfig
_elementtree        doctest             optparse            tabnanny
_functools          dumbdbm             os                  tarfile
_hashlib            dummy_thread        os2emxpath          telnetlib
_heapq              dummy_threading     parser              tempfile
_io                 email               pdb                 termios
_json               encodings           pickle              textwrap
_locale             ensurepip           pickletools         this
_multiprocessing    errno               pipes               thread
_osx_support        exceptions          pkg_resources       threading
_pyio               fcntl               pkgutil             time
_random             filecmp             platform            timeit
_sha256             fileinput           plistlib            toaiff
_sha512             fnmatch             png                 token
_socket             formatter           popen2              tokenize
_sre                formencode          poplib              trace
_ssl                fpformat            posix               traceback
_strptime           fractions           posixfile           tty
_struct             ftplib              posixpath           types
_symtable           functools           pprint              unicodedata
_sysconfigdata      future_builtins     profile             unittest
_threading_local    gc                  pstats              urllib
_warnings           genericpath         pty                 urllib2
_weakref            getopt              pwd                 urlparse
_weakrefset         getpass             py_compile          user
abc                 gettext             pyclbr              uu
aifc                glob                pydoc               uuid
antigravity         gzip                pydoc_data          warnings
anydbm              hashlib             pyexpat             wave
argparse            heapq               quopri              weakref
array               hmac                random              webbrowser
ast                 hotshot             re                  whichdb
asynchat            htmlentitydefs      reportlab           wsgiref
asyncore            htmllib             repr                xdrlib
atexit              httplib             requests            xml
audiodev            httplib2            resource            xmllib
babel               ihooks              rexec               xmlrpclib
base64              imaplib             rfc822              xxsubtype
bdb                 imghdr              rlcompleter         zipfile
beaker              imp                 robotparser         zipimport
binascii            importlib           runpy               zlib
binhex              imputil             sched
bisect              inspect             select

Enter any module name to get more help.  Or, type "modules spam" to search
for modules whose descriptions contain the word "spam".

>>>

[johndr@HO-SPLUNKES lib-dynload]$ pwd
/opt/splunk/lib/python2.7/lib-dynload
[johndr@HO-SPLUNKES lib-dynload]$ ldd _hashlib.so
        linux-vdso.so.1 =>  (0x00007fff5cfa9000)
        libssl.so.1.0.0 => not found
        libcrypto.so.1.0.0 => not found
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f80556f1000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f805535d000)
        /lib64/ld-linux-x86-64.so.2 (0x00000038e2a00000)

[johndr@HO-SPLUNKES lib-dynload]$ sudo which python
[sudo] password for johndr:
/usr/bin/python

[johndr@HO-SPLUNKES lib-dynload]$ pwd
/usr/lib64/python2.6/lib-dynload

[johndr@HO-SPLUNKES lib-dynload]$ ldd _hashlib.so
        linux-vdso.so.1 =>  (0x00007ffcd2c8f000)
        libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007f3b3bd82000)
        libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007f3b3b99e000)
        libpython2.6.so.1.0 => /usr/lib64/libpython2.6.so.1.0 (0x00007f3b3b5f7000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f3b3b3da000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f3b3b046000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f3b3ae01000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f3b3ab1a000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f3b3a916000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f3b3a6e9000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f3b3a4e5000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f3b3a2cf000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007f3b3a0cb000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f3b39e47000)
        /lib64/ld-linux-x86-64.so.2 (0x00000038e2a00000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f3b39c3c000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f3b39a38000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f3b3981e000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f3b395fe000)

I am stumped. Any assistance would be greatly appreciated.

Thanks!

MuS · ‎03-21-2018

Hi jodros,

what I usually do when I test Python scripts in Splunk, is to run Python the exact same way Splunk runs Python.
Also, in your example you just start /opt/splunk/bin/splunk, but what about the environment variables? ¯\_(ツ)_/¯

Here is what I do:

Start a shell like Splunk does: /opt/splunk/bin/splunk cmd /bin/bash
In this session start Python like Splunk does, and start to troubleshoot your python: /opt/splunk/bin/splunk cmd /opt/splunk/bin/python

I'm not saying that will fix your problem, but might help 😉

cheers, MuS

nmclaughl1 · ‎01-17-2019

Thanks MuS, I appreciate learning what you say Splunk does to execute scripts in $SPLUNK_HOME/etc/apps/my_TA_stuff/bin/*.py ...I hope to waste less time pre-flight testing with "hashlib md5 no has sha1" and "ldd _hashlib.so" errors, I followed the method you described and $SPLUNK_HOME/bin/python help("modules") stopped complaining and the scripts executed fine with Splunk's 2.7.14 python, system OpenSSL, and my copied modules from system site-packages. Huge thanks!

grumble · ‎03-20-2018

You might have some luck setting the LD_LIBRARY_PATH before python runs. Note the example below imports the requests package, which spits up the same error.

$ /opt/splunk/bin/python
Python 2.7.13 (default, Nov  1 2017, 03:43:17) 
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> ERROR:root:code for hash md5 was not found.
[snip]
>>> exit()
$ export LD_LIBRARY_PATH=/opt/splunk/lib
$ /opt/splunk/bin/python
Python 2.7.13 (default, Nov  1 2017, 03:43:17) 
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>

Not sure what to do (other than setting os.environ['LD_LIBRARY_PATH'] and spawning a new python process or something equally ugly) if you can only do things within python.

,You might have some luck setting the LD_LIBRARY_PATH before python runs:

$ /opt/splunk/bin/python
Python 2.7.13 (default, Nov  1 2017, 03:43:17) 
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> ERROR:root:code for hash md5 was not found.
[snip]
>>> exit()
$ export LD_LIBRARY_PATH=/opt/splunk/lib
$ /opt/splunk/bin/python
Python 2.7.13 (default, Nov  1 2017, 03:43:17) 
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>

Not sure what to do (other than setting os.environ['LD_LIBRARY_PATH'] and spawning a new python process or something equally ugly) if you can only do things within python.

doksu · ‎03-20-2018

Having the same issue with Splunk 7.0.1 on Fedora 27.

guarisma · ‎02-13-2017

I'm having this issue as well on my Splunk 6.4.x on Linux as well. I couldn't reproduce on my Splunk 6.5.x on Mac OS though.

[igor@splunk01 ca]$ /opt/splunk/bin/python
Python 2.7.11 (default, Mar 16 2016, 03:24:08)
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/opt/splunk/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/opt/splunk/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/splunk/lib/python2.7/site-packages/requests/__init__.py", line 58, in <module>
    from . import utils
  File "/opt/splunk/lib/python2.7/site-packages/requests/utils.py", line 25, in <module>
    from .compat import parse_http_list as _parse_list_header
  File "/opt/splunk/lib/python2.7/site-packages/requests/compat.py", line 7, in <module>
    from .packages import chardet
  File "/opt/splunk/lib/python2.7/site-packages/requests/packages/__init__.py", line 3, in <module>
    from . import urllib3
  File "/opt/splunk/lib/python2.7/site-packages/requests/packages/urllib3/__init__.py", line 16, in <module>
    from .connectionpool import (
  File "/opt/splunk/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 36, in <module>
    from .connection import (
  File "/opt/splunk/lib/python2.7/site-packages/requests/packages/urllib3/connection.py", line 43, in <module>
    from .util import (
  File "/opt/splunk/lib/python2.7/site-packages/requests/packages/urllib3/util/__init__.py", line 10, in <module>
    from .ssl_ import (
  File "/opt/splunk/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py", line 2, in <module>
    from hashlib import md5, sha1
ImportError: cannot import name md5

baker987 · ‎10-26-2017

I am having the exact same issue. Any updates ?

jodros · ‎04-18-2016

Bump. Any assistance would be appreciated. Thanks

jodros · ‎04-15-2016

Something else interesting, looks like libssl.so.1.0.0 and libcrypto.so.1.0.0 are in the parent directory:

[johndr@HO-SPLUNKES lib]$ pwd
/opt/splunk/lib
[johndr@HO-SPLUNKES lib]$ ls
copyright.txt libarchive.so.13.1.2 libbz2.so libexslt.so libmongoc-1.0.so libpcre.so libsqlite3.so libxml2.so libxmlsec1-openssl.so libxslt.so libz.so.1.2.3
engines libarchive.so.2 libbz2.so.1 libexslt.so.0 libmongoc-1.0.so.0 libpcre.so.0 libsqlite3.so.0 libxml2.so.2 libxmlsec1-openssl.so.1 libxslt.so.1 libz.so.1.2.8
libarchive.so libarchive.so.2.7.0 libbz2.so.1.0.3 libexslt.so.0.8.15 libmongoc-1.0.so.0.0.0 libpcre.so.0.0.1 libsqlite3.so.0.8.6 libxml2.so.2.7.6 libxmlsec1-openssl.so.1.2.20 libxslt.so.1.1.26 node_modules
libarchive.so.12 libbson-1.0.so libcrypto.so libexslt.so.0.8.17 libmongoc-priv.so libpcre.so.1 libssl.so libxml2.so.2.7.8 libxmlsec1.so libxslt.so.1.1.28 python2.6
libarchive.so.12.0.2 libbson-1.0.so.0 libcrypto.so.0.9.8 libjemalloc.so libmongoc-priv.so.0 libpcre.so.1.2.1 libssl.so.0.9.8 libxml2.so.2.9.1 libxmlsec1.so.1 libz.so python2.7
libarchive.so.13 libbson-1.0.so.0.0.0 libcrypto.so.1.0.0 libjemalloc.so.1 libmongoc-priv.so.0.0.0 libpcre.so.1.2.5 libssl.so.1.0.0 libxml2.so.2.9.2 libxmlsec1.so.1.2.20 libz.so.1 with_stats

Why am I getting "ERROR:root:code for hash md5 was not found." running a Python script on a search head?

New This Month - Splunk Observability updates and improvements for faster ...

What's New in Splunk Cloud Platform 9.3.2411?

Buttercup Games: Further Dashboarding Techniques (Part 6)