REST API Modular Input: How to split out modular REST input into multiple events?

Guitaraholis — Tue, 21 Mar 2017 20:50:28 GMT

I have a modular call made every 5 minutes to a performance checking API that validates all of our microservices are performing ok. Currently in our lab its onboarded as a single one line event.

{"https://web-call-one.co.uk":{"TimeStarted":"2017-03-21T14:57:26.638684208Z","TimeStartedUnix":1490108246,"TimeToLastByte":19,"ContentLength":15928,"TimeToFirstByte":19,"EndpointURL":"https://web-call-one.co.uk","SpeedInKB":1690.118726383912,"TimeToConnect":1,"HTTPStatusCode":200},"https://web-call-two.co.uk":{"TimeStarted":"2017-03-21T14:57:52.981022715Z","TimeStartedUnix":1490108272,"TimeToLastByte":3,"ContentLength":1184637,"TimeToFirstByte":1,"EndpointURL":"https://web-call-two.co.uk","SpeedInKB":49103.969662105934,"TimeToConnect":0,"HTTPStatusCode":200},"https://web-call-three.co.uk":{"TimeStarted":"2017-03-21T14:57:26.022008698Z","TimeStartedUnix":1490108246,"TimeToLastByte":19,"ContentLength":127135,"TimeToFirstByte":15,"EndpointURL":"https://web-call-three.co.uk","SpeedInKB":3224.4794945448634,"TimeToConnect":1,"HTTPStatusCode":200}}

if i change the format in an editor these are what the distinct events look like....

    "https://web-call-one.co.uk": {
        "TimeStarted": "2017-03-21T14:57:26.638684208Z",
        "TimeStartedUnix": 1490108246,
        "TimeToLastByte": 19,
        "ContentLength": 15928,
        "TimeToFirstByte": 19,
        "EndpointURL": "https://web-call-one.co.uk",
        "SpeedInKB": 1690.118726383912,
        "TimeToConnect": 1,
        "HTTPStatusCode": 200
    },
    "https://web-call-two.co.uk": {
        "TimeStarted": "2017-03-21T14:57:52.981022715Z",
        "TimeStartedUnix": 1490108272,
        "TimeToLastByte": 3,
        "ContentLength": 1184637,
        "TimeToFirstByte": 1,
        "EndpointURL": "https://web-call-two.co.uk",
        "SpeedInKB": 49103.969662105934,
        "TimeToConnect": 0,
        "HTTPStatusCode": 200
    },
    "https://web-call-three.co.uk": {
        "TimeStarted": "2017-03-21T14:57:26.022008698Z",
        "TimeStartedUnix": 1490108246,
        "TimeToLastByte": 19,
        "ContentLength": 127135,
        "TimeToFirstByte": 15,
        "EndpointURL": "https://web-call-three.co.uk",
        "SpeedInKB": 3224.4794945448634,
        "TimeToConnect": 1,
        "HTTPStatusCode": 200
    },

ideally i'd like to have these be ingested as separate events.... LINE_BREAKER won't play ball and i've tried a few of the custom handlers but can't quite get them worked out. this is what i tried to no avail at present...

class MyRestHandler:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
        if response_type == "json":
            output = json.loads(raw_response_output)
            for event in output["TimeStarted"]:
                print_xml_stream(json.dumps(event))
        else:
            print_xml_stream(raw_response_output)

Any assistance would be great

Re: REST API Modular Input: How to split out modular REST input into multiple events?

woodcock — Sat, 25 Mar 2017 06:17:52 GMT

Try this:

LINE_BREAKER = (^{|},|}})(?:"https|\w*$)
SHOULD_LINEMERGE = false

topic Re: REST API Modular Input: How to split out modular REST input into multiple events? in All Apps and Add-ons

REST API Modular Input: How to split out modular REST input into multiple events?

Re: REST API Modular Input: How to split out modular REST input into multiple events?