All Apps and Add-ons
Highlighted

How to troubleshoot why REST API Modular Input stopped working and how to create my own response handler to split the array returned?

Path Finder

Hi,

I've installed this app in my Splunk 6.2.3 instance running on Windows and have been able to use it to set up a few data inputs, for example here is one pulling in data using the flightaware.com FlightXML API - returns recent flight info for a particular aircraft 'ident':

[rest://FlightXML VH-FVF]
auth_password = ____________
auth_type = basic
auth_user = ________
endpoint = http://flightxml.flightaware.com/json/FlightXML2/FlightInfoEx
host = VH-FVF
http_method = GET
index = flightaware
index_error_response_codes = 0
polling_interval = 60
response_type = json
sourcetype = flightxml
streaming_request = 0
url_args = ident=VHFVF

I've got a few of these working - all the same except I've changed the host and url_args values. These are returning and indexing an array of flights.

I thought I'd have a go at setting up my own response handler to parse and split the array returned so that each element of the flights array is indexed individually. Somehow in doing this I think I've broken the one input in the example below, but all the others work ok still. I've tried removing and re-adding the input, reset $SPLUNKHOME$\etc\apps\rest_ta\bin\responsehandlers.py and $SPLUNKHOME$\etc\apps\rest_ta\bin\responsehandlers.pyc to default, restarted Splunk and even restarted the whole server but still no new events are getting indexed.

Can someone please offer some advice on how to troubleshoot this?

Also, I'd love some advice on how to create my own response handler to split the array. I'm new to python.

Here is an example of the raw output indexed at the moment in line with the input above:

{"FlightInfoResult":{"next_offset":15,"flights":[{"ident":"FD531","aircrafttype":"PC12","filed_ete":"01:21:00","filed_time":1432678027,"filed_departuretime":1432684800,"filed_airspeed_kts":269,"filed_airspeed_mach":"","filed_altitude":170,"route":"DCT YTHI RIKAB N640 AD","actualdeparturetime":1432688340,"estimatedarrivaltime":1432690260,"actualarrivaltime":1432690360,"diverted":"","origin":"YTHI","destination":"YPAD","originName":"","originCity":"","destinationName":"Adelaide Int'l","destinationCity":"Adelaide, South Australia AU"},{"ident":"FD531","aircrafttype":"PC12","filed_ete":"00:55:00","filed_time":1432678011,"filed_departuretime":1432679400,"filed_airspeed_kts":271,"filed_airspeed_mach":"","filed_altitude":180,"route":"DCT AD J15 YORKE","actualdeparturetime":1432680900,"estimatedarrivaltime":1432683480,"actualarrivaltime":1432683480,"diverted":"","origin":"YPAD","destination":"YTHI","originName":"Adelaide Int'l","originCity":"Adelaide, South Australia AU","destinationName":"","destinationCity":""},{"ident":"FD520","aircrafttype":"PC12","filed_ete":"00:41:00","filed_time":1432657590,"filed_departuretime":1432661700,"filed_airspeed_kts":266,"filed_airspeed_mach":"","filed_altitude":160,"route":"DCT REN V395 BLACK V454 AD","actualdeparturetime":1432662120,"estimatedarrivaltime":1432664460,"actualarrivaltime":1432664449,"diverted":"","origin":"YREN","destination":"YPAD","originName":"Renmark","originCity":"Renmark, South Australia AU","destinationName":"Adelaide Int'l","destinationCity":"Adelaide, South Australia AU"},{"ident":"FD520","aircrafttype":"PC12","filed_ete":"00:19:00","filed_time":1432657590,"filed_departuretime":1432658700,"filed_airspeed_kts":269,"filed_airspeed_mach":"","filed_altitude":170,"route":"DCT AD V361 SEDAN V324 REN","actualdeparturetime":1432658520,"estimatedarrivaltime":1432660260,"actualarrivaltime":1432660260,"diverted":"","origin":"YPAD","destination":"YREN","originName":"Adelaide Int'l","originCity":"Adelaide, South Australia AU","destinationName":"Renmark","destinationCity":"Renmark, South Australia AU"},{"ident":"FD520","aircrafttype":"PC12","filed_ete":"00:19:00","filed_time":1432641249,"filed_departuretime":1432655400,"filed_airspeed_kts":269,"filed_airspeed_mach":"","filed_altitude":170,"route":"DCT YTBB DCT RIKAB N640 AD","actualdeparturetime":1432651860,"estimatedarrivaltime":1432653720,"actualarrivaltime":1432653707,"diverted":"","origin":"YTBB","destination":"YPAD","originName":"Tumby Bay","originCity":"Tumby Bay, South Australia AU","destinationName":"Adelaide Int'l","destinationCity":"Adelaide, South Australia AU"},{"ident":"FD520","aircrafttype":"PC12","filed_ete":"01:00:00","filed_time":1432641231,"filed_departuretime":1432642200,"filed_airspeed_kts":271,"filed_airspeed_mach":"","filed_altitude":180,"route":"DCT AD J15 YORKE DCT","actualdeparturetime":1432643760,"estimatedarrivaltime":1432646340,"actualarrivaltime":1432646186,"diverted":"","origin":"YPAD","destination":"YTBB","originName":"Adelaide Int'l","originCity":"Adelaide, South Australia AU","destinationName":"Tumby Bay","destinationCity":"Tumby Bay, South Australia AU"},{"ident":"FD523","aircrafttype":"PC12","filed_ete":"02:00:00","filed_time":1432622082,"filed_departuretime":1432629900,"filed_airspeed_kts":270,"filed_airspeed_mach":"","filed_altitude":220,"route":"DCT MTG V259 AD","actualdeparturetime":1432633680,"estimatedarrivaltime":1432637400,"actualarrivaltime":1432637400,"diverted":"","origin":"YMTG","destination":"YPAD","originName":"Mount Gambier","originCity":"Mount Gambier, South Australia AU","destinationName":"Adelaide Int'l","destinationCity":"Adelaide, South Australia AU"},{"ident":"FD523","aircrafttype":"PC12","filed_ete":"00:49:00","filed_time":1432622082,"filed_departuretime":1432622700,"filed_airspeed_kts":269,"filed_airspeed_mach":"","filed_altitude":230,"route":"DCT AD W519 MTG","actualdeparturetime":1432623283,"estimatedarrivaltime":1432626240,"actualarrivaltime":1432626240,"diverted":"","origin":"YPAD","destination":"YMTG","originName":"Adelaide Int'l","originCity":"Adelaide, South Australia AU","destinationName":"Mount Gambier","destinationCity":"Mount Gambier, South Australia AU"},{"ident":"FD523","aircrafttype":"PC12","filed_ete":"01:37:00","filed_time":1432606716,"filed_departuretime":1432611900,"filed_airspeed_kts":266,"filed_airspeed_mach":"","filed_altitude":150,"route":"DCT MARGO H84 AD","actualdeparturetime":1432616400,"estimatedarrivaltime":1432618140,"actualarrivaltime":1432618133,"diverted":"","origin":"YPIR","destination":"YPAD","originName":"Port Pirie","originCity":"Port Pirie, South Australia AU","destinationName":"Adelaide Int'l","destinationCity":"Adelaide, South Australia AU"},{"ident":"FD523","aircrafttype":"PC12","filed_ete":"00:42:00","filed_time":1432606716,"filed_departuretime":1432610100,"filed_airspeed_kts":242,"filed_airspeed_mach":"","filed_altitude":60,"route":"","actualdeparturetime":1432612260,"estimatedarrivaltime":1432612800,"actualarrivaltime":1432612800,"diverted":"","origin":"YBOC","destination":"YPIR","originName":"Booleroo Centre","originCity":"Booleroo Centre South Australia AU","destinationName":"Port Pirie","destinationCity":"Port Pirie, South Australia AU"},{"ident":"FD523","aircrafttype":"PC12","filed_ete":"00:57:00","filed_time":1432606716,"filed_departuretime":1432607400,"filed_airspeed_kts":271,"filed_airspeed_mach":"","filed_altitude":180,"route":"DCT AD","actualdeparturetime":1432609140,"estimatedarrivaltime":1432611300,"actualarrivaltime":1432611300,"diverted":"","origin":"YPAD","destination":"YBOC","originName":"Adelaide Int'l","originCity":"Adelaide, South Australia AU","destinationName":"Booleroo Centre","destinationCity":"Booleroo Centre South Australia AU"},{"ident":"FD512","aircrafttype":"PC12","filed_ete":"00:33:00","filed_time":1432522032,"filed_departuretime":1432530000,"filed_airspeed_kts":272,"filed_airspeed_mach":"","filed_altitude":200,"route":"DCT YKIG DCT RUVUS V259 AD","actualdeparturetime":1432529880,"estimatedarrivaltime":1432532340,"actualarrivaltime":1432532340,"diverted":"","origin":"YKIG","destination":"YPAD","originName":"Kingston","originCity":"Kingston, South Australia AU","destinationName":"Adelaide Int'l","destinationCity":"Adelaide, South Australia AU"},{"ident":"FD512","aircrafttype":"PC12","filed_ete":"00:33:00","filed_time":1432522032,"filed_departuretime":1432524600,"filed_airspeed_kts":272,"filed_airspeed_mach":"","filed_altitude":190,"route":"DCT AD W519 LRT DCT","actualdeparturetime":1432524863,"estimatedarrivaltime":1432526880,"actualarrivaltime":1432526880,"diverted":"","origin":"YPAD","destination":"YKIG","originName":"Adelaide Int'l","originCity":"Adelaide, South Australia AU","destinationName":"Kingston","destinationCity":"Kingston, South Australia AU"},{"ident":"FD536","aircrafttype":"PC12","filed_ete":"00:59:00","filed_time":1432278114,"filed_departuretime":1432292400,"filed_airspeed_kts":269,"filed_airspeed_mach":"","filed_altitude":170,"route":"DCT WHA H84 AD","actualdeparturetime":1432294080,"estimatedarrivaltime":1432296300,"actualarrivaltime":1432296300,"diverted":"","origin":"YWHA","destination":"YPAD","originName":"Whyalla","originCity":"Whyalla, South Australia AU","destinationName":"Adelaide Int'l","destinationCity":"Adelaide, South Australia AU"},{"ident":"FD536","aircrafttype":"PC12","filed_ete":"01:15:00","filed_time":1432278114,"filed_departuretime":1432285200,"filed_airspeed_kts":269,"filed_airspeed_mach":"","filed_altitude":170,"route":"DCT PLC DCT WHA","actualdeparturetime":1432287922,"estimatedarrivaltime":1432290120,"actualarrivaltime":1432290120,"diverted":"","origin":"YPLC","destination":"YWHA","originName":"Port Lincoln","originCity":"Port Lincoln, South Australia AU","destinationName":"Whyalla","destinationCity":"Whyalla, South Australia AU"}]}}

I'd like to split out each 'flight' element and index them individually. Ideally the timestamp of each event would just be the time the event was indexed.

Thanks a lot!

Highlighted

Re: How to troubleshoot why REST API Modular Input stopped working and how to create my own response handler to split the array returned?

Path Finder

Also, when I clone the input that is not working and set it to a different (test) index it works ok.

0 Karma
Highlighted

Re: How to troubleshoot why REST API Modular Input stopped working and how to create my own response handler to split the array returned?

Path Finder

Found this in splunkd.log:

06-02-2015 01:17:35.951 +0930 ERROR ExecProcessor - message from "python "C:\Program Files\Splunk\etc\apps\rest_ta\bin\rest.py"" Exception performing request: HTTPConnectionPool(host='flightxml.flightaware.com', port=80): Max retries exceeded with url: /json/FlightXML2/FlightInfoEx?ident=VHFVF (Caused by <class 'socket.error'>: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond)
0 Karma
Highlighted

Re: How to troubleshoot why REST API Modular Input stopped working and how to create my own response handler to split the array returned?

Path Finder

I've tried adding my own response handler to responsehandlers.py with the aim of getting each flight element indexed indiviudally as follows:

class FlightXMLHandler:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
        if response_type == "json":
            flightxmlrawdata = json.loads(raw_response_output)

            flightxmlfirstlevel = flightxmlrawdata['FlightInfoExResult']

            flightxmlflights = flightxmlfirstlevel['flights']

            for flight in flightxmlflights:
                print_xml_stream(json.dumps(flight))
        else:
            print_xml_stream(raw_response_output)

I based this on the following python code I got working offline - outputs each flight element as a separate line:

import json
flightxmlrawresponse = 'PASTED RAW XML EXAMPLE ABOVE HERE'
flightxmlrawdata  = json.loads(flightxmlrawresponse)
flightxmlfirstlevel =  flightxmlrawdata['FlightInfoExResult']
flightxmlflights = flightxmlfirstlevel['flights']
for flight in flightxmlflights:
    print json.dumps(flight)

Doesn't seem to work though - any ideas where I'm going wrong?

0 Karma
Highlighted

Re: How to troubleshoot why REST API Modular Input stopped working and how to create my own response handler to split the array returned?

Ultra Champion

Try this response handler , just add the following class to responsehandlers.py and then declare it to be applied in your REST setup

class FlightInfoEventHandler:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
        if response_type == "json":        
            output = json.loads(raw_response_output)
            for flight in output["FlightInfoResult"]["flights"]:
                print_xml_stream(json.dumps(flight)) 


        else:
            print_xml_stream(raw_response_output) 

alt text

View solution in original post

Highlighted

Re: How to troubleshoot why REST API Modular Input stopped working and how to create my own response handler to split the array returned?

Path Finder

Excellent, thanks for that. I just tried adding that class to responsehandlers.py like I did the other day with my less elegant attempt. I restarted Splunk and also tried disabling and enabling the rest_ta Splunk app, but having specified that response handler in a REST input that does was indexing events using the default handler, no further events are getting indexed. I also tried adding a whole new input and specifying the new handler and a new index just for testing but no events get indexed.
Any ideas please?
Many thanks

0 Karma
Highlighted

Re: How to troubleshoot why REST API Modular Input stopped working and how to create my own response handler to split the array returned?

Path Finder

I just also tried this on a different Splunk instance - running on Mac OS X rather than Windows. On the Mac OS X instance the events are getting indexed the way I want after specifying the new custom response handler.
Could these issues be caused due to running on Windows?
Thanks

0 Karma
Highlighted

Re: How to troubleshoot why REST API Modular Input stopped working and how to create my own response handler to split the array returned?

Path Finder

Nevermind - I made a typo which I've corrected and it is all working now. Thanks a lot for your help.

0 Karma