Solved: A bytes like object is required, not string - Splu... - Page 2

hakon_nor · ‎01-07-2020

Hi,

When I try to set the JRE Installation path(java_home) during configuration of Splunk DB connect 3.2.0 I get the following error; "A bytes like object is required, not string". I'm currently using Splunk Enterprise 8.0.1, with python 3.7 set as the default python version. This error only occurs when I use python 3.7, using python 2.7 removes the problem.

The whole problem seems to originate from how the different python versions handles strings, as described by Splunk(I dont have enough karma to post links so ill just quote it):

"In Python 2, a native, default string is a binary or "bytes" string. In Python 3, a native string is instead a unicode string. Python 3 is strict about not mixing different types of strings. Often, an explicit conversion is needed or a runtime error is produced.

In Python 2, default and binary strings shared the same type but in Python 3, you now must keep the two string types distinctly separated. Binary data should not be stored in a default string since Python 3 defaults to unicode strings, not binary."

So my question is how do I convert the string to a bytes like object?

hakon_nor · ‎01-10-2020

Posting the solution to the original question given in the comments for visibility:

The solution was to change jre_validator.py under splunk_app_db_connect/bin/dbx2 to take into account the different string types. Splunk docs provided the solution, replace the file with the following code:

#! /usr/bin/env python
import os
from .dbx_logger import logger
import threading
import subprocess
import traceback
import shlex
from . import C
import sys
from six import string_types

class Command(object):
    """
    Enables to run subprocess commands in a different thread with TIMEOUT option.

    Based on jcollado's solution:
    http://stackoverflow.com/questions/1191374/subprocess-with-timeout/4825933#4825933
    """
    command = None
    process = None
    status = None
    output, error = '', ''

    def __init__(self, command):
        # string_types is a basestring() in Python 2 and str in Python 3
        if isinstance(command, string_types):
            # split the command using shell like syntax
            command = shlex.split(command)
        self.command = command


    def run(self, timeout=None, **kwargs):
        """ Run a command then return: (status, output, error). """
        def target(**kwargs):
            try:
                self.process = subprocess.Popen(self.command, **kwargs)
                self.output, self.error = self.process.communicate()
                self.status = self.process.returncode
            except Exception as ex:
                self.error = traceback.format_exception_only(type(ex), ex)
                self.status = -1
        # default stdout and stderr
        if 'stdout' not in kwargs:
            kwargs['stdout'] = subprocess.PIPE
        if 'stderr' not in kwargs:
            kwargs['stderr'] = subprocess.PIPE
        # thread
        thread = threading.Thread(target=target, kwargs=kwargs)
        thread.start()
        thread.join(timeout)
        if thread.is_alive():
            self.process.terminate()
            thread.join()
        return self.status, self.output, self.error


def matchesAny(line, values):
    return line.lstrip().startswith(tuple(values))

def validateJRE(javaCmd):
    attrs = {}
    reason = ""
    operation = "validate java command: {}.".format(javaCmd)

    if os.name == "nt":
        jreInfoCmd = '"%s" %s' % (javaCmd, C.JRE_INFO_OPTIONS)
    else:
        jreInfoCmd = '%s %s' % (javaCmd, C.JRE_INFO_OPTIONS)
    c = Command(jreInfoCmd)
    retval, output, error = c.run(10)
    if sys.version_info[0] == 3:
        output = output.decode('utf-8')
        error = error.decode('utf-8')

    if retval != 0:
        reason = error
        logger.debug(error)
        # smartly parse the error if we can.
        for line in error:
            if line.startswith("OSError:") or line.startswith("WindowsError"):
                reason = line
        isValid = False
        logger.critical(reason)
    else:
        pairs = [ line.lstrip().split(' = ', 2) for line in error.splitlines() if matchesAny(line, C.JRE_WANTED_KEYS)]

        for pair in pairs:
            k, v = pair
            attrs[k] = v

        version = attrs.get(C.JRE_VERSION_KEY, '')
        vendor = attrs.get(C.JRE_VENDOR_KEY, '')
        vm = attrs.get(C.JRE_VM_KEY, '')

        isValid = (version == C.JRE_WANT_VERSION and (vm.startswith(C.JRE_WANT_VM_ORACLEJDK) or vm.startswith(C.JRE_WANT_VM_OPENJDK)))
        if not isValid:
            reason = {"message": "Unsupported JRE detected",
                      "jre_using": "Using %s JRE version %s, %s" % (vendor, version, vm),
                      "jre_need": "Need Oracle Corporation JRE version 1.8 or OpenSDK 1.8"}
            logger.critical(reason)

    details = [str(reason), operation]

    return isValid, " ".join(details)

if os.name == 'nt':
    JAVA_DEPENDENCIES = [os.path.join("bin", "java.exe"),
                         os.path.join("bin", "keytool.exe")]

else:
    JAVA_DEPENDENCIES = [os.path.join("bin", "java"),
                         os.path.join("bin", "keytool")]

def checkDependencies(javaHome):
    reason = ""
    for dep in JAVA_DEPENDENCIES:
        fullPath = os.path.join(javaHome, dep)
        if not os.path.exists(fullPath):
            reason = "Missing JRE dependency: %s" % fullPath
            logger.critical(reason)
            return False, reason
    return True, reason

View solution in original post

A bytes like object is required, not string - Splunk DB Connect

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Announcing the General Availability of Splunk Enterprise Security 8.1!

Developer Spotlight with William Searle

Are you a member of the Splunk Community?

A bytes like object is required, not string - Splunk DB Connect

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Announcing the General Availability of Splunk Enterprise Security 8.1!

Developer Spotlight with William Searle