topic Re: alert to remotely run a script in Alerting

alert to remotely run a script

tmarlette — Wed, 16 Dec 2015 22:50:21 GMT

I have created a python script in order to ssh to a remote machine and run a script on that machine to unlock user accounts.
I am only getting an return code 255.

I have eliminated all 'splunk code' from the script, and the python script by itself works just fine when ran, and unlocks the account on the remote machine. I am wondering what I am doing wrong.

I have also copied the appropriate .ssh key to the remote machines in order to remove authentication when ssh to the remote machine.

This is my script:

'''

import sys
import splunk.Intersplunk
import subprocess
import logging
import os

# System logging
logger = logging.getLogger('testssh')
hdlr = logging.FileHandler('/tmp/testssh.txt')
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
logger.setLevel(logging.INFO)

try:

    keywords,options = splunk.Intersplunk.getKeywordsAndOptions() # Get all the options passed

    # Check for passed parameters
    if not options.has_key('host'):
        splunk.Intersplunk.generateErrorResults("no host specified")
        exit(0)
    if not options.has_key('user'):
        splunk.Intersplunk.generateErrorResults("no user specified")
        exit(0)
   if not options.has_key('command'):
        splunk.Intersplunk.generateErrorResults("no command specified")
        exit(0)

    command = options.get('command', None)
    host = options.get('host', None)
    user = options.get('user', None)

    results,dummyresults,settings = splunk.Intersplunk.getOrganizedResults()
    for r in results:
        try:
            # Call the script passing all the necessary arguments
            p = subprocess.Popen(["ssh -i /idn/home/tmarlett/.ssh/id_rsa -q -t -t tmarlett@r[host] r[command]"],stdin=subprocess.PIPE, stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
            # Get the output from script and push it to new output column
            out,err = p.communicate()
            #logger.info(out.decode("utf-8"))
            r["output"]= out.decode("utf-8")
            r["error"]=err
            r["return_code"]=p.returncode;
        except ValueError, e:
            results = splunk.Intersplunk.generateErrorResults(str(e))
        except OSError, e:
            results = splunk.Intersplunk.generateErrorResults(str(e))

    #Output results back to Splunk
    splunk.Intersplunk.outputResults(results)

except Exception, e:
    results = splunk.Intersplunk.generateErrorResults(str(e))

And this is the output it shows me when running the script:

Does anyone have any insight as to why this would be happening?

Re: alert to remotely run a script

jkat54 — Thu, 17 Dec 2015 01:33:03 GMT

Hello,

Your script needs a shebang

 #!/usr/local/python

Re: alert to remotely run a script

jkat54 — Thu, 17 Dec 2015 01:34:54 GMT

 which python

To get the proper path

Re: alert to remotely run a script

jkat54 — Thu, 17 Dec 2015 01:37:49 GMT

You'll also want to look at other apps on splunkbase to see how they implement the built in "splunk" python logger....

 _logger....

Re: alert to remotely run a script

jkat54 — Thu, 17 Dec 2015 01:39:28 GMT

Error 255 could coming from the subprocess too because subprocess returns 0-255.

Re: alert to remotely run a script

MuS — Thu, 17 Dec 2015 01:41:18 GMT

Or read the docs http://docs.splunk.com/Documentation/Splunk/6.3.1/AdvancedDev/ModInputsLog on how to set up logging in python scripts

Re: alert to remotely run a script

jkat54 — Thu, 17 Dec 2015 02:18:21 GMT

there's a way to log it through splunkd.log or your own, etc. for more details than even Splunk docs offer, just see python docs for logging. https://docs.python.org/2/library/logging.html

Re: alert to remotely run a script

tmarlette — Thu, 17 Dec 2015 15:37:46 GMT

I wish that was the case, however I have the shebang in there, and it's still doing this. I just left it out of my answer. Sorry for the confusion.

Re: alert to remotely run a script

tmarlette — Thu, 17 Dec 2015 16:40:15 GMT

Truthfully, I believe that is what this is. I say that because, when I adjust the syntax of the subprocess, I can see some of the output in splunk, so the script itself looks like it's working. For whatever reason It just doesn't like want to use the splunk search results accordingly.

also, when I do run the script with these settings, I also see a failed login attempt in the destination machine's syslog logs. here is the message:

Failed password for tmarlett from port 35168 ssh2

Re: alert to remotely run a script

tmarlette — Thu, 17 Dec 2015 16:41:07 GMT

mind you, the EXACT same line of the code is SUCCESSFUL when put into a python only script.

Re: alert to remotely run a script

tmarlette — Wed, 06 Jan 2016 16:51:51 GMT

Can you recommend a good app to use as a reference for logging?

Re: alert to remotely run a script

tmarlette — Thu, 02 Jun 2016 21:06:00 GMT

I figured this out. This was an issue with passing ssh keys properly. Make sure the script you're using is passing the appropriate ssh key of the user that is running the Splunk instance on that search head.