I know this is an old one, but I went searching in the hopes someone had already written, so sharing here. This is specifically for EntraID (/Azure AD) Audit logs - Which are "valid" json, however there's some wierd stuff going on within the JSON itself - And a real mixture of data structures (probably depends on which developer wrote each piece i'm guessing). Anyway, hope it helps someone - it deals with the 2x value data as well as the name/old/new type data. index=entra_audit activityDisplayName="Consent to application" category="ApplicationManagement" ``` Entra ID Audit Logs - Extract Application Consent JSON into usable fields ``` | spath operationType | rename targetResources{}.* AS targetResources.* | table _time index sourcetype source correlationId additionalDetails* activityDisplayName category operationType result initiatedBy.user.ipAddress initiatedBy.user.userPrincipalName targetResources.* ``` additionalDetails is a 2 entry key/value pair in JSON ``` | rename additionalDetails{}.* AS additionalDetails_* | eval additionalDetails=mvzip(additionalDetails_key,additionalDetails_value, "=") | mvexpand additionalDetails | eval Key=mvindex(split(additionalDetails,"="),0), Value=mvindex(split(additionalDetails,"="),1) | eval additionalDetails.{Key}=Value ``` targetResources.modifiedProperties{} is a 3 entry name/newvalue/oldvalue in JSON ``` | rename targetResources.modifiedProperties{}.* AS targetm_* | eval targetm=mvzip(targetm_displayName,targetm_newValue, ";") | eval targetm=mvzip(targetm,targetm_oldValue, ";") | mvexpand targetm | eval Key=mvindex(split(targetm,";"),0).".newValue", Value=trim(mvindex(split(targetm,";"),1),"\"") | eval modifiedProperties.{Key}=Value | eval Key=mvindex(split(targetm,";"),0).".oldValue", Value=trim(mvindex(split(targetm,";"),2),"\"") | eval modifiedProperties.{Key}=Value ``` Now extract the permissions, which are in yet a different format - multiple fields with `double_bracket`old => `double_bracket`new, but json format is the same old/new format as other above fields - yet the old and new is stored in the "newValue" field only, and "oldValue" is blank``` ``` Within the double brackets are multiple ID's which relate to individual scopes, however we're only interested in the scope and context type ``` ``` Remove spaces ``` | rex mode=sed field=modifiedProperties.ConsentAction.Permissions.newValue "s/ //g" ``` Extract new/old values ``` | rex field=modifiedProperties.ConsentAction.Permissions.newValue "\[(?P<perms_new>\[.*\])\]=>\[(?P<perms_old>\[.*\])\]" | makemv perms_new delim="]," | mvexpand perms_new | eval perms_new = trim(perms_new,"[]") | rex field=perms_new "ConsentType:(?P<perms_type_new>.*?),.*Scope:(?P<perms_scope_new>.*?)," | makemv perms_old delim="]," | mvexpand perms_old | eval perms_old = trim(perms_old,"[]") | rex field=perms_old "ConsentType:(?P<perms_type_old>.*?),.*Scope:(?P<perms_scope_old>.*?)," | rename perms_type_new AS permissions.consentType.newValue perms_scope_new AS permissions.scope.newValue | rename perms_type_old AS permissions.consentType.oldValue perms_scope_old AS permissions.scope.oldValue ``` Clean up unecessary fields - Slighly faster to do this before the transaction``` | fields - Key Value *_key *_value targetm* additionalDetails modifiedProperties.ConsentAction.Permissions* perms* ``` Condense those events down to a single event ``` | transaction correlationId maxspan=1s ``` Format into a nicer table and field names ``` | rename initiatedBy.user.ipAddress AS src initiatedBy.user.userPrincipalName AS user | rename modifiedProperties.ConsentContext.* AS context.* | rename modifiedProperties.TargetId.* AS target.* | table _time index sourcetype source category operationType result src user activityDisplayName correlationId additionalDetails.* targetResources.* target.* context.* permissions.* ```| collect index=my_summary ``` For performance I suggest you collect that into a summary index, csv or kvstore Here's a sanitised copy of an event: {"id": "Directory_zzzzz-zzzz-zzzz", "category": "ApplicationManagement", "correlationId": "zzzz-zzzz-zzzz-zzzz-zzzz", "result": "success", "resultReason": "", "activityDisplayName": "Consent to application", "activityDateTime": "2024-08-05T06:10:19.1808273Z", "loggedByService": "Core Directory", "operationType": "Assign", "initiatedBy": {"app": null, "user": {"id": "zzzz-zzzz-zzzz-zzzz-zzzz", "displayName": null, "userPrincipalName": "zzzz", "ipAddress": "0.0.0.0", "userType": null, "homeTenantId": null, "homeTenantName": null}}, "targetResources": [{"id": "zzzz-zzzz-zzzz-zzzz-zzzz", "displayName": "Microsoft Graph PowerShell", "type": "ServicePrincipal", "userPrincipalName": null, "groupType": null, "modifiedProperties": [{"displayName": "ConsentContext.IsAdminConsent", "oldValue": null, "newValue": "\"True\""}, {"displayName": "ConsentContext.IsAppOnly", "oldValue": null, "newValue": "\"False\""}, {"displayName": "ConsentContext.OnBehalfOfAll", "oldValue": null, "newValue": "\"False\""}, {"displayName": "ConsentContext.Tags", "oldValue": null, "newValue": "\"WindowsAzureActiveDirectoryIntegratedApp\""}, {"displayName": "ConsentAction.Permissions", "oldValue": null, "newValue": "\"[] => [[Id: zzzz-zzzz-zzzz-zzzz-zzzz_-OGPWDcaCSZ3yquwoJpK3, ClientId: zzzz-zzzz-zzzz-zzzz-zzzz, PrincipalId: zzzz-zzzz-zzzz-zzzz-zzzz, ResourceId: zzzz-zzzz-zzzz-zzzz-zzzz, ConsentType: Principal, Scope: Application.ReadWrite.All Organization.Read.All AuditLog.Read.All openid profile offline_access, CreatedDateTime: , LastModifiedDateTime ]]; \""}, {"displayName": "TargetId.ServicePrincipalNames", "oldValue": null, "newValue": "\"zzzz-zzzz-zzzz-zzzz-zzzz\""}]}], "additionalDetails": [{"key": "User-Agent", "value": "EvoSTS"}, {"key": "AppId", "value": "zzzz-zzzz-zzzz-zzzz-zzzz"}]}
... View more