topic Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value in Dashboards & Visualizations

How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

rubans — Mon, 28 Sep 2020 19:47:54 GMT

For e.g:
For the following response how do I check XML or JSON for a status & succeeded elements/props (also never used python so any example would help):
XML:

FIN_UK_TB_AVAILABLE
2015-03-06
2015-03-26T11:02:43.905Z
Success
true

MAJOR
2015-03-06T12:00:00.000Z

JSON:
{
"_id": "2015-03-06_FIN_UK_TB_AVAILABLE",
"event": "FIN_UK_TB_AVAILABLE",
"date": "2015-03-06T00:00:00.000Z",
"raised": "2015-03-26T11:02:43.905Z",
"status": "Success",
"succeeded": true,
"states": [],
"escalations": [
{
"path": "MAJOR",
"date": "2015-03-06T12:00:00.000Z"
}
],
"dependencies": [],
"_updateCount": 1,
"_created": "2015-03-26T08:20:24.979Z"
}

Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

Damien_Dallimor — Mon, 28 Sep 2020 19:43:21 GMT

I presume you are referring to this app : https://splunkbase.splunk.com/app/1546/ ?

If so , there are a bunch of example response handlers here for processing JSON repsonses :

https://github.com/damiendallimore/SplunkModularInputsPythonFramework/blob/master/implementations/rest/bin/responsehandlers.py

1) write your custom handler and add it to SPLUNK_HOME/rest_ta/bin/responsehandlers.py

class SomeCustomHandler:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):       

        if response_type == "json":        
            output = json.loads(raw_response_output)

            #get the status and succeeded attributes from the json response
            status = output["status"]
            succeeded = output["succeeded"]

            #presumably then use them for some sort of boolean logic

            #output json event to Splunk
            print_xml_stream(json.dumps(output))

        else:
            print_xml_stream(raw_response_output)

2) declare this custom handler in your stanza setup

Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

rubans — Mon, 28 Sep 2020 19:48:17 GMT

Ok, it seems now the Splunk is not even logging the request/response for the JSON content.
I don't see any entries in my search results for the expected data point.
When the response is :
896487

This is logged in search results

However, when the response is:

FIN_UK_TB_AVAILABLE
2015-03-06
2015-03-26T11:02:43.905Z
Success
true

FIN_UK_CURRENT_TB
batchId
55196a640cf2dafa21580e27

FIN_UK_CURRENT_TB
rowCount
316400

FIN_UK_CURRENT_TB
checksum
52336917775281.00

FIN_UK_PRIOR_TB
batchId
551985b80cf2dafa21580e29

FIN_UK_PRIOR_TB
priorStatus
Open

FIN_UK_PRIOR_TB
rowCount
149366

FIN_UK_PRIOR_TB
checksum
24119627122230.60

MAJOR
2015-03-06T12:00:00.000Z

It's not being logged
I don't see any errors in Splunk System monitor associated with this either.
It seems to be very inconsistent, is there something it doesn't like about response from the REST service that may stop logging the result?
This is before I have even applied the Custom Handler.

Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

Damien_Dallimor — Fri, 01 May 2015 11:42:17 GMT

I just gave you a generic example to point you in the right direction. I know nothing about your REST endpoint , JSON payload or the logic you are trying to apply to the JSON response.
So, the intent was to give you a example which you could then build something off.

Any errors will be searchable in Splunk with : index=_internal ExecProcessor error rest.py

Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

rubans — Fri, 01 May 2015 13:00:16 GMT

I had a look and see nothing relevant, who can help me resolve this issue?
I'm evaluating this product so would need to understand why the response is not even being logged for this particular request.

Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

rubans — Fri, 01 May 2015 15:22:12 GMT

It doesn't seem to monitor the url as expected, either that or it's not logging the result.
Can you let me know how I can look into this issue or provide the customer service number?

Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

Damien_Dallimor — Fri, 01 May 2015 17:38:02 GMT

OK , lets do troubleshooting 101....

1) can you see the request hitting your server (logs) ?
2) can you see the request on the wire (wireshark is useful) ?
3) what is your REST config on the Splunk side (look at inputs.conf with basic shell searching)
4) any firewalls ?

Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

rubans — Wed, 06 May 2015 11:08:23 GMT

HI Damien,
1& 2) Yes I can see both type of requests/responses using wireshark every 60 seconds as expected
3)Can you provide more detail? what kind of scripting? I don't see any inputs.conf file in the C:\Program Files\Splunk\etc\apps\rest_ta folder.
4)I don't think firewall should be an issue as I can see the responses coming back and also I can see the logging as mentioned for one response in Splunky (both HTTP endpoints are being served from the same Host domain)

Re: How to use Custom Response Handlers for monitoring REST service to check for an XML element or JSON value

Damien_Dallimor — Sat, 09 May 2015 04:38:32 GMT

1) Go to SPLUNK_HOME/etc/apps and do a filesystem search under this directory for the stanza you setup for your REST Input , it gets saved to a file names inputs.conf , and then post this stanza for me to look at ?

2) Have you written a custom response handler ? what does this code look like ? Maybe you have an error in your code ?

3) When you search for events in Splunk , how are you searching for them ? Search command , time range etc.. ?