How to parse JSON (blob) data by time stamp AND clean up the trailing brackets and commas?

[Log_wrangler]

So I am using this stanza in my .../local props.conf

[mscs:storage:blob]
SHOULD_LINEMERGE = true
LINE_BREAKER: (,[\r\n]+\s+){
TRUNCATE = 0
KV_MODE = json

This works fine to parse my events like this

"time": "2018-07-09T16:14:21.3832528Z",
             "resourceId": "blah",
             "operationName": "blah blah",
             "category": "blah",
             "resultType": "blah",
             "resultSignature": "blah",
             "durationMs": 1234,
             "callerIpAddress": "x.x.x.x",
             "correlationId": "x-x-x-x-x",
             "identity": {"blah blah":{"blah"}},
             "level": "blah",
             "location": "blah",
             "properties": {"statusCode":"OK","serviceRequestId":"x-x-x"}
        }
    ]
}

OR

"time": "2018-07-09T16:14:21.3832528Z",
             "resourceId": "blah",
             "operationName": "blah blah",
             "category": "blah",
             "resultType": "blah",
             "resultSignature": "blah",
             "durationMs": 1234,
             "callerIpAddress": "x.x.x.x",
             "correlationId": "x-x-x-x-x",
             "identity": {"blah blah":{"blah"}},
             "level": "blah",
             "location": "blah",
             "properties": {"statusCode":"OK","serviceRequestId":"x-x-x"}
                }
        ,
        {

Does anyone know how to drop the brackets and commas?

Thank you

[jconger]

For JSON in a blob, these props.conf setting work pretty well for most use cases:

[sourcetype]
LINE_BREAKER = \}([\r\n]\s*,[\r\n]\s*)\{
SEDCMD-remove_header = s/\{\s*\"records\"\:\s*\[\s*//g
SEDCMD-remove_footer = s/\][\r\n]\s*\}.*//g
SHOULD_LINEMERGE = false
KV_MODE = json
TIME_PREFIX = time\":\"

[Log_wrangler]

Thank you for the reply, I will give it a try and let you know.

[coccyx]

Where is this from? This is pretty printed JSON data, it would be easier to fix the source to send JSON as single line events.

[Log_wrangler]

They are log files send to blob to be pulled into Splunk and ingested. Unfortunately I don't have access to the source to send in a perfect format.

Do you know a way to discard the brackets and commas?

[coccyx]

Do you have a sample of the full log, not how Splunk parsed it?

We need to see how multiple events look.

[Log_wrangler]

Unfortunately I don't have access to the source logs, and probably would not be able to post the original without redacting a lot of sensitive info.

What I do know is that some of the events in the blob are Azure WAF logs.

Would you be able to use that as a source sample?

Here is a link that has a number of the sample json logs
https://docs.microsoft.com/en-us/azure/application-gateway/application-gateway-diagnostics

I appreciate your efforts.
thank you