Developing for Splunk Enterprise
Highlighted

Custom command - Getinfo probe failed

Explorer

Hi!
I created a custom command for a comparison between json.
The steps to create it:

0) created fieldcompare folder in etc/apps (with bin, metadata and default subfolders)

1) fieldcompare.py (in etc/apps/fieldcompare/bin)

# custom command

import splunk.Intersplunk  
import json
from splunklib.searchcommands import \
     dispatch, StreamingCommand, Configuration, Option, validators


 # long def function (searchleveljson) that looks for things in jsons
 #etc etc

(isgetinfo, sys.argv) = splunk.Intersplunk.isGetInfo(sys.argv)
args, kwargs = splunk.Intersplunk.getKeywordsAndOptions()

if isgetinfo:
     # streaming, generating, retevs, reqsop, preop
    splunk.Intersplunk.outputInfo(True, False, False, False, None)


(results, dummyresults, settings) = splunk.Intersplunk.getOrganizedResults()

field1 = kwargs.get("field1", "field1")
field2 = kwargs.get("field2", "field2")

try:
    for result in results:
        try:
            j1 = json.load(field1)
            j2 = json.load(field2)
        except KeyError:
             # If either field is missing, simply ignore
            continue

        resultKey = list()
        resultValues = list()
        searchleveljson(j1,j2,resultKey,resultValues)

        result["mismatched keys"]=json.dumps(resultKey)
        result["Value diff"]=json.dumps(resultValues)


splunk.Intersplunk.outputResults(results)

2) commands.conf (in etc/apps/fieldcompare/default)

[fieldcompare]
filename = fieldcompare.py
supports_getinfo = true

3) default.meta (in etc/apps/fieldcompare/metadata)

 [commands/fieldcompare]
 access = read : [ * ], write : [ admin ]
 export = system

 [scripts/fieldcompare.py]
 access = read : [ * ], write : [ admin ]
 export = system

4) copy splunklib (with modularinput and searchcommands) in etc/apps/fieldcompare/bin from etc\apps\framework\contrib\splunk-sdk-python

5) Restart splunk

6) index=* etc etc | table expected actual | fieldcompare field1=expected field2=actual

but I get an error that says:

Error in 'script': Getinfo probe failed for external search command 'fieldcompare'

What am I doing wrong?

Thanks,
have a good day.

Debora

0 Karma
Highlighted

Re: Custom command - Getinfo probe failed

Motivator

Are you getting any error when you start your splunk server? Any compilation error in your customcommand will get reported only at starting of Splunk server.

Also you can check https://splunkbase.splunk.com/app/3447/. It has different way of writing streaming command.

0 Karma
Highlighted

Re: Custom command - Getinfo probe failed

Explorer

it doesn't find any errors at startup.
This is right method to use intersplunk?

0 Karma
Highlighted

Re: Custom command - Getinfo probe failed

Champion

I would suggest moving to the SDK (referenced in your script as splunklib but not actually used). Check out the filter.py example that comes with the SDK for guidance.

0 Karma
Highlighted

Re: Custom command - Getinfo probe failed

Motivator

Yes use of intersplunk is correct.

Few things I just noticed from your code. You are creating tupple when fetching results. You should update that query as follows:

results, dummyresults, settings = splunk.Intersplunk.getOrganizedResults()

When you return result, it should be list of directories.

0 Karma
Highlighted

Re: Custom command - Getinfo probe failed

Explorer

I tried to create the custom command in both ways.

Intersplunk way:
I added "import sys" and removed import from splunklib.
From splunk search page I read an error that asks to add GETINFO or EXECUTE
I changed the command to:
| fieldcompare EXECUTE field1=x field2=y
but I get a generic error 1.
I add getOrganizedResults before outputResults but nothing has changed.
I would just like to pass two parameters and get two new ones for each line but it seems more complicated than expected.

SDK way:
I followed the documentation.
I started from SDK templates, taking inspiration from the examples and I created a class that extends a GeneratingCommand but the error is always the same:

Error in 'script': Getinfo probe failed for external search command 'jsoncompare'

0 Karma
Highlighted

Re: Custom command - Getinfo probe failed

Explorer

Thanks 🙂
I solved the problem with Intersplunk!

0 Karma
Highlighted

Re: Custom command - Getinfo probe failed

Champion

Please post your solution as an answer and accept that answer so that others may benefit from your findings.

Highlighted

Re: Custom command - Getinfo probe failed

Explorer

Intersplunk solution:

apps/fieldcompare/bin/fieldcompare.py:

# custom command script

import splunk.Intersplunk  
import json
import sys

def is_json(myjson):
    return type(myjson) == type({})

#found difference between two jsons 
def searchleveljson(json1,json2,resultKey,resultValues):
    keys=set(json1.keys())
    keys2=set(json2.keys())

    keys.update(keys2)
    for json_event in keys:
        if json_event not in json1:
            resultKey.append(json_event)
            resultValues.append("actual."+json_event+"."+json2[json_event])
            continue;
        if json_event not in json2:
            resultKey.append(json_event)
            resultValues.append("expected."+json_event+"."+json1[json_event])
            continue;
        v1 = json1[json_event]
        v2 = json2[json_event]
        if v1 != v2:
            if is_json(v1) and is_json(v2):
                searchleveljson(v1,v2,resultKey,resultValues)
            else:
                resultKey.append(json_event)
                diff = "actual."+json_event+"="+json2[json_event]
                diff+= " "
                diff+= "expected."+json_event+"="+json1[json_event]
                resultValues.append(diff)

(isgetinfo, sys.argv) = splunk.Intersplunk.isGetInfo(sys.argv)
args, kwargs = splunk.Intersplunk.getKeywordsAndOptions()

if isgetinfo:
    # streaming, generating, retevs, reqsop, preop
    splunk.Intersplunk.outputInfo(True, False, False, False, None)


results, dummyresults, settings = splunk.Intersplunk.getOrganizedResults()

field1 = kwargs.get("json1", "field1")
field2 = kwargs.get("json2", "field2")


for result in results:
    try:
        #corrected errors for converting string to json
        #json.load(fieldx) -> json.loads(result[fieldx])
        j1 = json.loads(result[field1])
        j2 = json.loads(result[field2])
    except ValueError:
        pass

    resultKey = list()
    resultValues = list()
    searchleveljson(j1,j2,resultKey,resultValues)

    result["mismatched_keys"]=json.dumps(resultKey)
    result["value_diff"]=json.dumps(resultValues)


splunk.Intersplunk.outputResults(results)

apps/fieldcompare/default/custom.conf:

[fieldcompare]
filename = fieldcompare.py
streaming=true

apps/fieldcompare/metadata/default.meta:

  [commands/fieldcompare]
  access = read : [ * ], write : [ admin ]
  export = system


 [scripts/fieldcompare.py]
 access = read : [ * ], write : [ admin ]
 export = system

I deleted all other folders (spunk and splunklib)

Restart splunk.

| table column1 column2 | fieldcompare __EXECUTE__ json1=column1 json2=column2

the result is a table with the following columns:
column1 | column2 | mismatchedkeys | valuediff

mismatchedkeys contains all different keys
value
diff contains all different value

View solution in original post

0 Karma