I'll start by saying I may be doing this completely wrong. 🙂 I need help removing the first 2 lines and the last 2 lines of a file via props and transforms. I have tried so far only works to remove the first to lines (so all events process properly except the last on in the file b/c the last 2 lines end up mucking up the json for that event). I have a JSON file (sample content below); the file starts with "value" : [ with several hundred objects in the values in that array.   { "value": [ { "properties": { "roleName": "Virtual Machine Administrator", "type": "CustomRole", "description": "administer and update virtual machines.", "assignableScopes": [ "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx" ], "permissions": [ { "actions": [ "Microsoft.Storage/*/read", "Microsoft.Compute/virtualMachines/performMaintenance/action" ], "notActions": [] } ], "createdOn": "2018-11-01T20:32:29.71317Z", "updatedOn": "2018-11-01T20:32:29.71317Z", "createdBy": "af5e3f18-3a18-4141-8296-5efb1b267cd9", "updatedBy": "af5e3f18-3a18-4141-8296-5efb1b267cd9" }, "id": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx/providers/Microsoft.Authorization/roleDefinitions/92e07475-99a8-4e12-9fc2-c4034be97904", "type": "Microsoft.Authorization/roleDefinitions", "name": "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx" }, { "properties": { "roleName": "Virtual Machine Support", "type": "CustomRole", "description": "Can administer and update virtual machines.", "assignableScopes": [ "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx", "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx" ], "permissions": [ { "actions": [ "Microsoft.Storage/*/read", "Microsoft.Compute/disks/delete", "Microsoft.Compute/disks/write", "Microsoft.Compute/snapshots/write", "Microsoft.Compute/disks/beginGetAccess/action" ], "notActions": [] } ], "createdOn": "2018-11-28T02:09:47.2262816Z", "updatedOn": "2020-09-14T17:33:57.5619979Z", "createdBy": "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx", "updatedBy": "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx" }, "id": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx/providers/Microsoft.Authorization/roleDefinitions/e74f813f-9dee-48f4-a0ba-ec37f07a95f9", "type": "Microsoft.Authorization/roleDefinitions", "name": "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx" } ] }     All a care about is what is in the array (as individual events in splunk). So I'd like to is strip off (at the beginning)   { "value": [    and remove the following from the end.   ] }   If I do that then everything I have works perfectly for splunk. My current problem is that my props and transforms will remove { "value": [ from the beginning but I can't seem to remove the ] } from the end.   ## props.conf [mscs:azure:roledef] TRANSFORMS-timestamp=timestampeval TRANSFORMS-elimL1=eliminateL1, eliminateLE KV_MODE = json LINE_BREAKER = (?ms)[\r\n]+\s{4}}(,[\n\r]+) NO_BINARY_CHECK = true SHOULD_LINEMERGE = false TRANSFORMS-timestamp = timestampeval TRUNCATE = 0 category = Structured description = A variant of the JSON source type, with support for nonexistent timestamps disabled = false pulldown_type = true ## transforms.conf [timestampeval] INGEST_EVAL = _time=strptime(replace(source,".*(?=\\\)\\\\",""),"Role Definitions_%Y-%m-%dT%H %M %S") [eliminateL1] REGEX = (?ms)^(?:{.+"value":\s\[.) DEST_KEY = queue FORMAT = nullQueue [eliminateLE] REGEX = (?ms)(?:\s+]\s})$ DEST_KEY = queue FORMAT = nullQueue     ... View more

I'm using the Azure Add-on for splunk to pull in our azure AD signin, audit and user data; all is work well for the most part with the exception of some user events (sourcetype="azure:aad:user") seem to have truncated json and therefore don't parse correctly. Is there a limit setting for this that can remediate this? ... View more