runshellscript search command does not return the results file ($8 argument)

[testadrianbelen]

This docs (https://docs.splunk.com/Documentation/SplunkCloud/6.6.0/SearchReference/Runshellscript) says $8 argument is the path where the search result file will be saved however after running the search command:

* | head 1 | runshellscript test.py [ search * | stats count | return count ] 2 3 4 5 6 7 /var/www

does not create a results.csv

[woodcock]

From my (unaccepted) answer here, quoted verbatim:
https://answers.splunk.com/answers/41949/passing-search-results-to-external-python-script.html

This isn't the "right" way to do this, but it is the most expedient and it allows you to recycle your traditional alert script for use in regular searches, too. This example assumes that you will be passing 2 fields to the scirpt: scr_ip and host; you will need to adjust slightly for the fields that you will be using.

Insert this code to your MyAlertScript.py code right before your existing code that accesses the results.csv.gz file in argv[8]:

# In order to facilitate Event Workflow Actions using runshellscript,
# we will hijack the arguments in one special case as follows:
#    1(sys.argv[1]) = '1'
#    2(sys.argv[2]) = '"<src_ip>","<host>"'
#    3(sys.argv[3]) = 'Hack'
#    4(sys.argv[4]) = 'to'
#    5(sys.argv[5]) = 'run'
#    7(sys.argv[6]) = 'from'
#    6(sys.argv[7]) = 'runshellscript'
#    8(sys.argv[8]) = * <- DO NOT CHECK because splunk modifieds this on the way in
# If in this format we will pull the data directly out of the 2nd argument,
# instead of out of the restults file.
specialCase = False # initialize to FALSE
if ((sys.argv[1] == '1') and (sys.argv[3] == 'Hack') and (sys.argv[4] == 'to') and
    (sys.argv[5] == 'run') and (sys.argv[6] == 'from') and
    (sys.argv[7] == 'runshellscript')):
    print "SPECIAL CASE!\n" # Special case!
    specialCase = True # make sure we delete this file at the end!

    fnz = workdir + 'results.csv.gz'
    with gzip.open(fnz, 'wb') as OFH:
       OFH.write('src_ip,host\n')
       OFH.write(sys.argv[2])
    OFH.close()

    sys.argv[8] = fnz
    #print "SPECIAL CASE: ARGV8=<" + sys.argv[8] + "> fnz=<" + fnz + ">\n"

Then at the bottom, add this, too:

if (specialCase): os.remove(sys.argv[8]) # delete fake zip file we made

Next you need a macro like this (to abstract away the trickery):

[MyScript]
definition = table src_ip host\
| map maxsearches=5000 search="|runshellscript MyAlertScript.py 1 \"\\\"\\\"$src_ip$\\\",\\\"$host$\\\"\\\"\" Hack to run from runshellscript 8"
iseval = 0

Now, to use it, you just do this:

My Search To Get Events With src_ip And host Here | `MyScript`

SPECIAL WARNING! This will not scale nicely if you pass a large number of results to the script because of the map command but it works GREAT for small numbers of events.

[jkat54]

Runshellscript is not supported as a search command. I honestly have no idea what it is for.

https://docs.splunk.com/Documentation/SplunkCloud/6.6.0/SearchReference/Runshellscript

Description
For Splunk Enterprise deployments, executes scripted alerts. This command is not supported as a search command.

Instead you would save a search such as * | head 1, and in trigger actions you would select "run a script" then you'd type in the name of your script that is in the bin folder of the app that the search is in.

Then i your script you be able to use $7, or $ARGSV[7] to find the path where splunk wrote the results.csv.

csvPath=$7
...code to read / manipulate data found in csvPath...

[testadrianbelen]

so it is just an environment variable that can be accessed or used in the script, but it can't be used as an argument in a script. http://docs.splunk.com/Documentation/SplunkCloud/latest/Alert/Configuringscriptedalerts

[jkat54]

I typically write custom search commands when I need access to the data in the search pipe. It's just easier and more versatile.

[jkat54]

Download my ta-webtools and look at the scripts in bin, and how I wore them up using commands.conf.

If you know Python it will be easy to reuse the code. If you don't know Python then it will be easier than starting from scratch but you can post the code and ask for help as needed.