topic Re: Custom command - Getinfo probe failed in Splunk Dev

Custom command - Getinfo probe failed

drebai — Mon, 11 Dec 2017 10:26:58 GMT

Hi!
I created a custom command for a comparison between json.
The steps to create it:

0) created fieldcompare folder in etc/apps (with bin, metadata and default subfolders)

1) fieldcompare.py (in etc/apps/fieldcompare/bin)

# custom command

import splunk.Intersplunk  
import json
from splunklib.searchcommands import \
     dispatch, StreamingCommand, Configuration, Option, validators


 # long def function (searchleveljson) that looks for things in jsons
 #etc etc

(isgetinfo, sys.argv) = splunk.Intersplunk.isGetInfo(sys.argv)
args, kwargs = splunk.Intersplunk.getKeywordsAndOptions()

if isgetinfo:
     # streaming, generating, retevs, reqsop, preop
    splunk.Intersplunk.outputInfo(True, False, False, False, None)


(results, dummyresults, settings) = splunk.Intersplunk.getOrganizedResults()

field1 = kwargs.get("field1", "field1")
field2 = kwargs.get("field2", "field2")

try:
    for result in results:
        try:
            j1 = json.load(field1)
            j2 = json.load(field2)
        except KeyError:
             # If either field is missing, simply ignore
            continue

        resultKey = list()
        resultValues = list()
        searchleveljson(j1,j2,resultKey,resultValues)

        result["mismatched keys"]=json.dumps(resultKey)
        result["Value diff"]=json.dumps(resultValues)


splunk.Intersplunk.outputResults(results)

2) commands.conf (in etc/apps/fieldcompare/default)

[fieldcompare]
filename = fieldcompare.py
supports_getinfo = true

3) default.meta (in etc/apps/fieldcompare/metadata)

 [commands/fieldcompare]
 access = read : [ * ], write : [ admin ]
 export = system

 [scripts/fieldcompare.py]
 access = read : [ * ], write : [ admin ]
 export = system

4) copy splunklib (with modularinput and searchcommands) in etc/apps/fieldcompare/bin from etc\apps\framework\contrib\splunk-sdk-python

5) Restart splunk

6) index=* etc etc | table expected actual | fieldcompare field1=expected field2=actual

but I get an error that says:

Error in 'script': Getinfo probe failed for external search command 'fieldcompare'

What am I doing wrong?

Thanks,
have a good day.

Debora

Re: Custom command - Getinfo probe failed

hardikJsheth — Mon, 11 Dec 2017 12:56:16 GMT

Are you getting any error when you start your splunk server? Any compilation error in your customcommand will get reported only at starting of Splunk server.

Also you can check https://splunkbase.splunk.com/app/3447/. It has different way of writing streaming command.

Re: Custom command - Getinfo probe failed

drebai — Mon, 11 Dec 2017 13:28:21 GMT

it doesn't find any errors at startup.
This is right method to use intersplunk?

Re: Custom command - Getinfo probe failed

micahkemp — Mon, 11 Dec 2017 23:51:53 GMT

I would suggest moving to the SDK (referenced in your script as splunklib but not actually used). Check out the filter.py example that comes with the SDK for guidance.

Re: Custom command - Getinfo probe failed

hardikJsheth — Wed, 13 Dec 2017 05:57:28 GMT

Yes use of intersplunk is correct.

Few things I just noticed from your code. You are creating tupple when fetching results. You should update that query as follows:

results, dummyresults, settings = splunk.Intersplunk.getOrganizedResults()

When you return result, it should be list of directories.

Re: Custom command - Getinfo probe failed

drebai — Wed, 13 Dec 2017 10:47:40 GMT

I tried to create the custom command in both ways.

Intersplunk way:
I added "import sys" and removed import from splunklib.
From splunk search page I read an error that asks to add GETINFO or EXECUTE
I changed the command to:
| fieldcompare EXECUTE field1=x field2=y
but I get a generic error 1.
I add getOrganizedResults before outputResults but nothing has changed.
I would just like to pass two parameters and get two new ones for each line but it seems more complicated than expected.

SDK way:
I followed the documentation.
I started from SDK templates, taking inspiration from the examples and I created a class that extends a GeneratingCommand but the error is always the same:

Error in 'script': Getinfo probe failed for external search command 'jsoncompare'

Re: Custom command - Getinfo probe failed

drebai — Wed, 13 Dec 2017 14:02:22 GMT

Thanks 🙂
I solved the problem with Intersplunk!

Re: Custom command - Getinfo probe failed

micahkemp — Wed, 13 Dec 2017 14:17:14 GMT

Please post your solution as an answer and accept that answer so that others may benefit from your findings.

Re: Custom command - Getinfo probe failed

drebai — Tue, 29 Sep 2020 17:17:01 GMT

Intersplunk solution:

apps/fieldcompare/bin/fieldcompare.py:

# custom command script

import splunk.Intersplunk  
import json
import sys

def is_json(myjson):
    return type(myjson) == type({})

#found difference between two jsons 
def searchleveljson(json1,json2,resultKey,resultValues):
    keys=set(json1.keys())
    keys2=set(json2.keys())

    keys.update(keys2)
    for json_event in keys:
        if json_event not in json1:
            resultKey.append(json_event)
            resultValues.append("actual."+json_event+"."+json2[json_event])
            continue;
        if json_event not in json2:
            resultKey.append(json_event)
            resultValues.append("expected."+json_event+"."+json1[json_event])
            continue;
        v1 = json1[json_event]
        v2 = json2[json_event]
        if v1 != v2:
            if is_json(v1) and is_json(v2):
                searchleveljson(v1,v2,resultKey,resultValues)
            else:
                resultKey.append(json_event)
                diff = "actual."+json_event+"="+json2[json_event]
                diff+= " "
                diff+= "expected."+json_event+"="+json1[json_event]
                resultValues.append(diff)

(isgetinfo, sys.argv) = splunk.Intersplunk.isGetInfo(sys.argv)
args, kwargs = splunk.Intersplunk.getKeywordsAndOptions()

if isgetinfo:
    # streaming, generating, retevs, reqsop, preop
    splunk.Intersplunk.outputInfo(True, False, False, False, None)


results, dummyresults, settings = splunk.Intersplunk.getOrganizedResults()

field1 = kwargs.get("json1", "field1")
field2 = kwargs.get("json2", "field2")


for result in results:
    try:
        #corrected errors for converting string to json
        #json.load(fieldx) -> json.loads(result[fieldx])
        j1 = json.loads(result[field1])
        j2 = json.loads(result[field2])
    except ValueError:
        pass

    resultKey = list()
    resultValues = list()
    searchleveljson(j1,j2,resultKey,resultValues)

    result["mismatched_keys"]=json.dumps(resultKey)
    result["value_diff"]=json.dumps(resultValues)


splunk.Intersplunk.outputResults(results)

apps/fieldcompare/default/custom.conf:

[fieldcompare]
filename = fieldcompare.py
streaming=true

apps/fieldcompare/metadata/default.meta:

  [commands/fieldcompare]
  access = read : [ * ], write : [ admin ]
  export = system


 [scripts/fieldcompare.py]
 access = read : [ * ], write : [ admin ]
 export = system

I deleted all other folders (spunk and splunklib)

Restart splunk.

| table column1 column2 | fieldcompare __EXECUTE__ json1=column1 json2=column2

the result is a table with the following columns:
column1 | column2 | mismatched_keys | value_diff

mismatched_keys contains all different keys
value_diff contains all different value