Developing for Splunk Enterprise
Highlighted

Use a different Python version with subprocess to create search script using custom command?

Path Finder

Purpose: Help the reader understand one approach to have Splunk kick-off a Python script that returns data to Splunk without indexing returned data, or modifying the Slunk Python installation.

Use Case: Splunk ships with a minimal implementation of Python 2.7. If you need to utilize a module that isn’t installed with the Splunk implementation it is recommended that an installation external to Splunk, with the modules installed, be used.

Expected Action: The user will initiate a Splunk search query using a custom command. This custom command will initiate a wrapper python script that will read a passed parameter/argument and initiate the target python script and return the values to the Slunk interface for display or use.

(Note: This is Splunk Enterprise on Windows Server)

Python Needs:

  1. Python Installation (external to Splunk Python installation) -Full Python or Virtual Environment.
  2. Install your required modules into your python environment (pip or other installation tool)
  3. Note the Python executable locations -Splunk Python location: (e.g. D:\Splunk\bin\python.exe) -External Python installation (e.g. D:\SplunkPythonvenv\p27_15\Scripts\python.exe)

Script Requirements:
1. Wrapper script: (see examplewrapper.py)
-Will work with Splunk python implementation. (No modules added outside of Splunk standard installation)
-Reference Intersplunk class
sys.path.append("/Splunk/Python-2.7/Lib/site-packages/splunk")
import Intersplunk
-Print output results from target script
2. Target script: (see exampletarget.py)
-Can use your modules or customizations
-Reference Intersplunk class
sys.path.append("/Splunk/Python-2.7/Lib/site-packages/splunk")
import Intersplunk
-Call Intersplunk.outputResults(results)
(results) is your result set and it must be in a format that can be consumed by Splunk.

Splunk Custom Search Command:
1. Create custom search command in commands.conf
[activatepython]
chuncked = false
filename = example_wrapper.py

Run Script from command line:

\Splunk\etc\apps\myapp\bin>........\bin\splunk cmd python examplewrapper.py scripttarget=exampletarget.py

Run Script from SPLalt text

Example Files:

######################################################################################
############### following is the example_wrapper.py file #############################
######################################################################################
import os
import subprocess
import re
import sys
sys.path.append("/Splunk/Python-2.7/Lib/site-packages/splunk")
import Intersplunk


custompythonpath = 'D:\Splunk_Python_venv\p27_15\Scripts\python.exe'
splunkpythonpath = 'D:\Splunk\bin\python.exe'

os.environ['PYTHONPATH'] = custompythonpath
# from splunk web interface this is needed to get the arguments
# example SPL "| scriptwrapper scripttarget=example_target.py"
keywords, argvals = Intersplunk.getKeywordsAndOptions()
try:
    #pull in the arguments
    scripttarget = argvals['scripttarget']
    #prevent code injection: verify we aren't getting a file outside of our run directory 
    fileVerify=re.split('/|\*|\\\\|\'|\"', scripttarget)
    if os.path.isfile(fileVerify[0]):
        #create full path to your script
        my_process = os.path.join(os.getcwd(), scripttarget)
        #create subprocess be sure to pass stdin, stdout, stderr
        p = subprocess.Popen([custompythonpath, my_process, splunkpythonpath], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        # get the output
        output = p.communicate()[0]
        # push it out
        print output
    else:
        print ('Unable to verify requested action'.format(fileVerify))
except Exception as e:
    print(e)


######################################################################################
############### following is the example_target.py file #############################
######################################################################################


# pypyodbc has a dependency on environment variable 'windir' be sure 
# if using the the activate process that the class passes in the evirnoment variable on activate.
import sys
import pypyodbc
import sys
import os
sys.path.append("/Splunk/Python-2.7/Lib/site-packages/splunk")
import Intersplunk

def GetSqlResults(connectionString, sqlText):
    results = []
    try:
        db = pypyodbc.connect(connectionString)
        cursor = db.cursor()
        cursor.execute(sqlText)
        columns = [column[0] for column in cursor.description]
        #had to do the following to get splunk to accept the result set (I thought this would be simpler)
        for row in cursor.fetchall():
            result = {}
            for column in columns:
                result[column] = row[column]
            results.append(result)
        db.close()
    except Exception as e:
        print('Bad things happened when executing SQL: {0}'.format(e))

    return results


if __name__ == "__main__":
    connectionString = "Trusted_Connection=yes; Driver={SQL Server};server=greatstuffserver;database=greatstuffdatabase'
    sqlText = 'select id, name from greatstufftable'
    results = []
    results = GetSqlResults(connectionString, sqlText)
    #send results to Intersplunk (the wrapper class will 'print' the output)
    Intersplunk.outputResults(results)
    sys.exit(0)

 

Labels (2)
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.