REST API Modular Input: How to make additional requests through a custom response handler?

[jeremiahMN]

I'm working with the rabbitMQ API and trying to make additional requests through a custom response handler.

We call the API/queues API which gives us all the queues. Then in the response handler, I'm trying to parse out the queue names and make additional requests to the api/queues/%2f/queuename api which gives detailed information regarding that queue.

Here's what I have in the response handler. I've removed all the other custom response handler class examples, etc... Any assistance would be greatly appreciated.

#add your custom response handler class to this module
import sys
import json
import datetime
import urlparse
import requests

import os.path

SPLUNK_HOME = os.environ.get("SPLUNK_HOME")

RESPONSE_HANDLER_INSTANCE = None
SPLUNK_PORT = 8089
STANZA = None
SESSION_TOKEN = None
REGEX_PATTERN = None

#dynamically load in any eggs in /etc/apps/snmp_ta/bin
EGG_DIR = SPLUNK_HOME + "/etc/apps/rest_ta/bin/"

for filename in os.listdir(EGG_DIR):
    if filename.endswith(".egg"):
        sys.path.append(EGG_DIR + filename)

#the default handler , does nothing , just passes the raw output directly to STDOUT
class DefaultResponseHandler:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
        cookies = response_object.cookies
        if cookies:
            req_args["cookies"] = cookies        
        print_xml_stream(raw_response_output)



class vueQueuesHandler:

    def __init__(self,**args):
        self.user = args['username']
        self.pw = args['password']
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):

        if response_type is not None and response_type=="json":

            output=json.loads(raw_response_output)

            for x in output:
                session = requests.session()
                session.auth = (self.user, self.pw)
                queueName = session.get('https://rabbitvts1dev.ourdomain.com/api/queues/%2f/' + x['name'], verify=False)
                queueJSON = json.loads(queueName.content)
                queueJSON['created_at'] = str(datetime.datetime.now())
                print_xml_stream(json.dumps(queueJSON))

        else:
            pass





#HELPER FUNCTIONS

# prints XML stream
def print_xml_stream(s):
    print "<stream><event unbroken=\"1\"><data>%s</data><done/></event></stream>" % encodeXMLText(s)



def encodeXMLText(text):
    text = text.replace("&", "&amp;")
    text = text.replace("\"", "&quot;")
    text = text.replace("'", "&apos;")
    text = text.replace("<", "<")
    text = text.replace(">", ">")
    text = text.replace("\n", "")
    return text

[Damien_Dallimor]

Guessing here , without being able to see the JSON , configuration stanza etc...

Does queueName.text rather than queueName.content work ?

Any errors in the logs ?

I don't know the rabbit rest api , but at a glance "%2f/" looks to be out of place.
http://hg.rabbitmq.com/rabbitmq-management/raw-file/3646dee55e02/priv/www-api/help.html

[jeremiahMN]

If I do this outside of the splunk system I get back the desired json data. So I'm guessing that queue.content works as well as the "%2f/" which is url encoded "/". The logging is pretty minimal, perhaps I'll need to add some additional logging to see what's going on.

import requests, json
from requests.auth import HTTPBasicAuth
from requests.packages.urllib3.exceptions import InsecureRequestWarning

requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

session = requests.session()
session.auth = ('username', 'password')
page = session.get('https://rabbitvts1dev.ourdomain.com/api/queues',verify=False)

jsonResponse = json.loads(page.content)

for x in jsonResponse:
    queueName = session.get('https://rabbitvts1dev.ourdomain.com/api/queues/%2f/' + x['name'], verify=False)
    print queueName.content