How to get Splunk to run my Python shell script?

[kt0703]

I am trying to use the Splunk Synthetic App in order to set up and monitor fake transactions that I create with Python.

I am running into problems when I try and get Splunk to run my shell script (Test2^SplunkOpenTest^.sh) which calls my Python program (Test2^SplunkOpenTest^.py) on a Linux VM. I go to Settings -> Data Inputs > Scripts and set the sourcetype and index to both be synthetic but somehow Splunk doesn’t collect the data even though the program works when I run it from the Linux terminal.

Because of this I looked inside the Splunkd.log file and noticed that every 60 seconds when Splunk is supposed to get the data from the script, it gives an error that looks like this:

06-20-2016 12:07:19.638 -0400 ERROR ExecProcessor – message from “/opt/splunk/etc/apps/splunk-app-synthetic/bin/Test2^SplunkOpenTest^.sh ImportError: No module named selenium

I have tried to figure this out for a day now and I feel like I’m missing something so simple but for the life of me I cannot figure out what that is.

My Test2^SplunkOpenTest^.sh file looks like so:

#!/bin/bash
unset PYTHONPATH
unset LD_LIBRARY_PATH
CLASSPATH=”/usr/lib/pyth/selenium-server-standalone-2.53.0.jar”
export CLASSPATH
SELENIUM_SERVER_JAR=”/usr/lib/pyth/selenium-server-standalone-2.53.0.jar”
export SELENIUM_SERVER_JAR
python /opt/splunk/etc/apps/splunk-app-synthetic/bin/Test2^SplunkOpenTest^.py

And my Test2^SplunkOpenTest^.py file looks like:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
import unittest, time, re
from splunktransactions import Transaction
class SplunkOpenTest(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Firefox()
        self.driver.implicitly_wait(30)
        self.base_url = "http://localhost:8000/en-US/account/login?return_to=%2Fen-US%2F"
    time.sleep(5)
        self.verificationErrors = []
        self.accept_next_alert = True
    def test_splunk_open(self):
        driver = self.driver
    a=Transaction(driver, 'Splunk') 
    a.TransactionStart(driver, 'Splunk Login Page') 
        driver.get(self.base_url + "/")
    a.TransactionEnd(driver, 'Splunk Login Page')
    time.sleep(4)
    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException as e: return False
        return True
    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException as e: return False
        return True
    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True
    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)
if __name__ == "__main__":
    unittest.main()

I appreciate the assistance with this!

[pswalia06]

Hi Guys,

I need help. Basically, if you see below dashboard topic name are amx and amx1 but the line chart is showing the line for both of them as one. How can we make them two lines instead of one? Please reply

[shaskell_splunk]

Running your script from the command line is not the same as running it from Splunk. When you run from the command line you're using the Python installation that ships with your Linux distribution. You need to test your script using the following command from the shell:

$SPLUNK_HOME/bin/splunk cmd python yourscript.py

If it runs from the shell with that command it should work in the app. The issue is most likely that the selenium Python module is not accessible to your script from the Python install that's bundled with Splunk. You'll need to build a Python egg file of selenium or put the dependent Python modules in:

/opt/splunk/etc/apps/splunk-app-synthetic/bin/

Have a look at the following answers post to see if it doesn't help.

https://answers.splunk.com/answers/220196/import-non-native-python-libraries-into-splunk.html

You can alternatively wriate a wrapper script that calls Python on your LInux server to execute your script. See here:

https://answers.splunk.com/answers/8/can-i-add-python-modules-to-the-splunk-environment.html

[MuS]

I agree on your answer in general; but reading the instruction of the App:

1- Install the app on the Splunk forwarder by extracting the tar file or doing it from the UI
2- Install Python 3.4 or above outside of the Splunk directory path on the same Splunk forwarder
3- Install selenium and user-agents modules using pip. To do so, you can follow the steps below:
    a- From the newly installed python directory , go to the bin folder and run the following commands:
    b- pip3.4 install user_agents
    c- pip3.4 install selenium

This one uses the OS python instead of the Splunk shipped Python. My assumption based on the error message is that selenium was not installed therefor the module is not available.

cheers, MuS

[shaskell_splunk]

I assumed the app used Splunk's Python. I should have checked the docs. Thanks for pointing that out. Yes, you seem to be missing selenium from the install of Python that the app is trying to use.