topic Re: Can a custom search script invoke an eval/where expression? in Splunk Search

Can a custom search script invoke an eval/where expression?

Lowell — Sat, 09 Oct 2010 00:22:37 GMT

I would like to be able to create a custom search command that uses an eval-expression as an argument. (Similar to how you can say ... | transaction startswith=(eval-expression) ...)

Does anyone know if it's possible to invoke an eval-expression from within the python custom search command? Obviously this can be done for built-in commands like transaction, but that doesn't seem to be written in python.

If this can't be done, any ideas as a functional workaround?

Re: Can a custom search script invoke an eval/where expression?

ziegfried — Sat, 09 Oct 2010 00:25:22 GMT

if it is not supported, it would be a great enhancement!

Re: Can a custom search script invoke an eval/where expression?

southeringtonp — Sat, 09 Oct 2010 00:26:56 GMT

You'd need to build the evaluation logic into your Python script.

Parameters like this are going to be passed into your Python script as command-line arguments -- you can get them from sys.argv and do whatever you want from there.

Re: Can a custom search script invoke an eval/where expression?

Lowell — Sat, 09 Oct 2010 00:49:37 GMT

Sorry, but I think your missing the point. I know I can do whatever I want within python, but the point is that I don't want to go build by own eval-engine. I simply want to leverage the one Splunk already built. (The eval command already has over 40 functions and that list is growing each release. I certainly don't want to go build my own mini expression language and all those functions. And then try to keep that in sync with every new splunk release.)

Re: Can a custom search script invoke an eval/where expression?

Lowell — Sat, 09 Oct 2010 03:05:23 GMT

Here is one hacked up way to do this. BTW, I'm going to let this unanswered for now, in the hopes that someone come up with a better (less-hackish) answer. Also, the ability to directly issue an eval/where expression from within a custom search command could be extended to many more situations than this limited hack.

After some playing around, I came up with an approach that works by using a combination of getinfo and a dynamic preop command. This approach leverages the fact that the getinfo mechanism calls the script twice, and passes in the arguments to the script both times. The first __GETINFO__ call dynamically builds a pre-streaming operation based on the argument passed to the script. This causes the pre-operation command to execute an eval command before my custom search command is run; and so the temporary field is accessible from my search command when it's called in __EXECUTE__ mode.

Below is a demo version of my script. If the "filter" eval expression evaluates to true, then function f1 is used to handle the event, and if it evaluates to false then function f2 is used instead. In this simple example, f1() adds two fields, and f2() multiples two fields. If no "filter" is given, then f2() is always called.

The script is called like this:

... | mycmd filter="date_hour>12" | table a b sum product

mycmd.py:

import splunk.Intersplunk

(isgetinfo, sys.argv) = splunk.Intersplunk.isGetInfo(sys.argv)
args, kwargs = splunk.Intersplunk.getKeywordsAndOptions()

# Process args
eval_filter = kvargs.get("filter", None)

if isgetinfo:
    # streaming, generating, retevs, reqsop, preop
    if eval_filter:
        splunk.Intersplunk.outputInfo(False, False, False, True, "addinfo | eval _filter=if(%s,1,0)" % eval_filter)
    else:
        splunk.Intersplunk.outputInfo(False, False, False, True, "addinfo")

def f1(r):
    # Add field 'a' and 'b'
    r["sum"] = int(r["a"]) + int(r["b"])
    return r

def f2(r):
    # Multiply field 'a' and 'b' 
    r["product"] = int(r["a"]) * int(r["b"])
    return r

if __name__ == '__main__':
    (results, dummyresults, settings) = splunk.Intersplunk.getOrganizedResults()
    if len(results) >= 1:
        search_id = results[0]["info_sid"]
    else:
        search_id = None
    logger.info("SID:  %r", search_id)
    output = [] 
    try:
        for result in results:
            if "_filter" in result:
                if result["_filter") == "1":
                    result = f1(result)
                else:
                    result = f2(result)
                del result["_filter"]
            else:
                result = f2(result)
            output.append(result)
        splunk.Intersplunk.outputResults(output)
    except Exception, e:
        splunk.Intersplunk.generateErrorResults("Unhandled exception:  %s" % (e,))

Re: Can a custom search script invoke an eval/where expression?

MuS — Mon, 14 Nov 2011 10:54:40 GMT

Hi Lowell, is this script working in 4.2.x? I'm trying to run it but it returns 'error 1' although I fixed your typo and change a|b field to existing ones that match my search