Splunk Search

o365:management:activity field extractions

adalbor
Builder

Has anyone had any success writing field extractions for O365 based events collected via the API?

The messages that are generated are HUGE and have multiple fields that contain multiple values.

I have tried to use eval and mvindex to see if its possible to extract those values but it doesn't appear to be working and I am wondering if its because of the JSON format.

Writing a regex for one of these events would have me ending up with something a page long lol.

Thanks,
Andrew

0 Karma

to4kawa
Ultra Champion
|makeresults
| eval _raw="[{\"CreationTime\":\"2015-06-29T20:03:19\",\"Id\":\"80c76bd2-9d81-4c57-a97a-accfc3443dca\",\"Operation\":\"PasswordLogonInitialAuthUsingPassword\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":9,\"ResultStatus\":\"failed\",\"UserKey\":\"1153977025279851686@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ClientIP\":\"134.170.188.221\",\"ObjectId\":\"admin@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"ExtendedProperties\":[{\"Name\":\"LoginError\",\"Value\":\"-2147217390;PP_E_BAD_PASSWORD;The entered and stored passwords do not match.\"}],\"Client\":\"Exchange\",\"LoginStatus\":-2147217390,\"UserDomain\":\"contoso.onmicrosoft.com\"},{\"CreationTime\":\"2015-06-29T20:03:34\",\"Id\":\"4e655d3f-35fa-42e0-b050-264b2d255c7a\",\"Operation\":\"PasswordLogonInitialAuthUsingPassword\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":9,\"ResultStatus\":\"success\",\"UserKey\":\"1153977025279851686@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ClientIP\":\"134.170.188.221\",\"ObjectId\":\"admin@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"Client\":\"Exchange\",\"LoginStatus\":0,\"UserDomain\":\"contoso.onmicrosoft.com\"},{\"CreationTime\":\"2015-06-29T20:04:55\",\"Id\":\"b567caf0-088e-4c1c-a4ea-633a1e3d66c8\",\"Operation\":\"Add User.\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":8,\"ResultStatus\":\"success\",\"UserKey\":\"1003BFFD8EC47CA6@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ObjectId\":\"user001@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"Actor\":[{\"ID\":\"1cef1fdb-ff52-48c4-8e4e-dfb5ea83d357\",\"Type\":2},{\"ID\":\"admin@contoso.onmicrosoft.com\",\"Type\":5},{\"ID\":\"1003BFFD8EC47CA6\",\"Type\":3}],\"ActorContextId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"InterSystemsId\":\"c2ced078-ad57-4079-a743-5c37f5284790\",\"IntraSystemId\":\"d1497f7e-15b4-49aa-83ad-11a17ca4a2f4\",\"Target\":[{\"ID\":\"user001@contoso.onmicrosoft.com\",\"Type\":5},{\"ID\":\"10037FFE91510806\",\"Type\":3}],\"TargetContextId\":\"41463f53-8812-40f4-890f-865bf6e35190\"}]"
| spath {} output=root
| stats count by root
| spath input=root Actor{} output=Actor
| mvexpand Actor
| spath input=Actor
| spath input=root Target{} output=Target
| mvexpand Target
| rename Type as Actor_Type
| spath input=Target
| spath input=root
| fields - Actor* root Target* count

https://docs.microsoft.com/en-Us/office/office-365-management-api/office-365-management-activity-api...
That's a lot of work.

If response is Array (not Object) like this, please make props.conf and LINE_BREAKER = (\[|,){\"CreationTime|(\]$)

0 Karma

adalbor
Builder

Thanks for the assistance. I tested that line_breaker config in our test environment but doesn't appear to be doing anything. It definitely is a lot of work to try and parse these logs. One example log I am looking at is over 32k characters!

0 Karma

to4kawa
Ultra Champion

Do you use SHOULD_LINEMERGE = true ?

0 Karma

to4kawa
Ultra Champion

can you provide sample logs link?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!