Splunk Search

o365:management:activity field extractions

adalbor
Builder

Has anyone had any success writing field extractions for O365 based events collected via the API?

The messages that are generated are HUGE and have multiple fields that contain multiple values.

I have tried to use eval and mvindex to see if its possible to extract those values but it doesn't appear to be working and I am wondering if its because of the JSON format.

Writing a regex for one of these events would have me ending up with something a page long lol.

Thanks,
Andrew

0 Karma

to4kawa
Ultra Champion
|makeresults
| eval _raw="[{\"CreationTime\":\"2015-06-29T20:03:19\",\"Id\":\"80c76bd2-9d81-4c57-a97a-accfc3443dca\",\"Operation\":\"PasswordLogonInitialAuthUsingPassword\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":9,\"ResultStatus\":\"failed\",\"UserKey\":\"1153977025279851686@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ClientIP\":\"134.170.188.221\",\"ObjectId\":\"admin@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"ExtendedProperties\":[{\"Name\":\"LoginError\",\"Value\":\"-2147217390;PP_E_BAD_PASSWORD;The entered and stored passwords do not match.\"}],\"Client\":\"Exchange\",\"LoginStatus\":-2147217390,\"UserDomain\":\"contoso.onmicrosoft.com\"},{\"CreationTime\":\"2015-06-29T20:03:34\",\"Id\":\"4e655d3f-35fa-42e0-b050-264b2d255c7a\",\"Operation\":\"PasswordLogonInitialAuthUsingPassword\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":9,\"ResultStatus\":\"success\",\"UserKey\":\"1153977025279851686@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ClientIP\":\"134.170.188.221\",\"ObjectId\":\"admin@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"Client\":\"Exchange\",\"LoginStatus\":0,\"UserDomain\":\"contoso.onmicrosoft.com\"},{\"CreationTime\":\"2015-06-29T20:04:55\",\"Id\":\"b567caf0-088e-4c1c-a4ea-633a1e3d66c8\",\"Operation\":\"Add User.\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":8,\"ResultStatus\":\"success\",\"UserKey\":\"1003BFFD8EC47CA6@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ObjectId\":\"user001@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"Actor\":[{\"ID\":\"1cef1fdb-ff52-48c4-8e4e-dfb5ea83d357\",\"Type\":2},{\"ID\":\"admin@contoso.onmicrosoft.com\",\"Type\":5},{\"ID\":\"1003BFFD8EC47CA6\",\"Type\":3}],\"ActorContextId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"InterSystemsId\":\"c2ced078-ad57-4079-a743-5c37f5284790\",\"IntraSystemId\":\"d1497f7e-15b4-49aa-83ad-11a17ca4a2f4\",\"Target\":[{\"ID\":\"user001@contoso.onmicrosoft.com\",\"Type\":5},{\"ID\":\"10037FFE91510806\",\"Type\":3}],\"TargetContextId\":\"41463f53-8812-40f4-890f-865bf6e35190\"}]"
| spath {} output=root
| stats count by root
| spath input=root Actor{} output=Actor
| mvexpand Actor
| spath input=Actor
| spath input=root Target{} output=Target
| mvexpand Target
| rename Type as Actor_Type
| spath input=Target
| spath input=root
| fields - Actor* root Target* count

https://docs.microsoft.com/en-Us/office/office-365-management-api/office-365-management-activity-api...
That's a lot of work.

If response is Array (not Object) like this, please make props.conf and LINE_BREAKER = (\[|,){\"CreationTime|(\]$)

0 Karma

adalbor
Builder

Thanks for the assistance. I tested that line_breaker config in our test environment but doesn't appear to be doing anything. It definitely is a lot of work to try and parse these logs. One example log I am looking at is over 32k characters!

0 Karma

to4kawa
Ultra Champion

Do you use SHOULD_LINEMERGE = true ?

0 Karma

to4kawa
Ultra Champion

can you provide sample logs link?

0 Karma
Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...