Splunk Search

o365:management:activity field extractions

adalbor
Builder

Has anyone had any success writing field extractions for O365 based events collected via the API?

The messages that are generated are HUGE and have multiple fields that contain multiple values.

I have tried to use eval and mvindex to see if its possible to extract those values but it doesn't appear to be working and I am wondering if its because of the JSON format.

Writing a regex for one of these events would have me ending up with something a page long lol.

Thanks,
Andrew

0 Karma

to4kawa
Ultra Champion
|makeresults
| eval _raw="[{\"CreationTime\":\"2015-06-29T20:03:19\",\"Id\":\"80c76bd2-9d81-4c57-a97a-accfc3443dca\",\"Operation\":\"PasswordLogonInitialAuthUsingPassword\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":9,\"ResultStatus\":\"failed\",\"UserKey\":\"1153977025279851686@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ClientIP\":\"134.170.188.221\",\"ObjectId\":\"admin@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"ExtendedProperties\":[{\"Name\":\"LoginError\",\"Value\":\"-2147217390;PP_E_BAD_PASSWORD;The entered and stored passwords do not match.\"}],\"Client\":\"Exchange\",\"LoginStatus\":-2147217390,\"UserDomain\":\"contoso.onmicrosoft.com\"},{\"CreationTime\":\"2015-06-29T20:03:34\",\"Id\":\"4e655d3f-35fa-42e0-b050-264b2d255c7a\",\"Operation\":\"PasswordLogonInitialAuthUsingPassword\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":9,\"ResultStatus\":\"success\",\"UserKey\":\"1153977025279851686@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ClientIP\":\"134.170.188.221\",\"ObjectId\":\"admin@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"Client\":\"Exchange\",\"LoginStatus\":0,\"UserDomain\":\"contoso.onmicrosoft.com\"},{\"CreationTime\":\"2015-06-29T20:04:55\",\"Id\":\"b567caf0-088e-4c1c-a4ea-633a1e3d66c8\",\"Operation\":\"Add User.\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":8,\"ResultStatus\":\"success\",\"UserKey\":\"1003BFFD8EC47CA6@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ObjectId\":\"user001@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"Actor\":[{\"ID\":\"1cef1fdb-ff52-48c4-8e4e-dfb5ea83d357\",\"Type\":2},{\"ID\":\"admin@contoso.onmicrosoft.com\",\"Type\":5},{\"ID\":\"1003BFFD8EC47CA6\",\"Type\":3}],\"ActorContextId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"InterSystemsId\":\"c2ced078-ad57-4079-a743-5c37f5284790\",\"IntraSystemId\":\"d1497f7e-15b4-49aa-83ad-11a17ca4a2f4\",\"Target\":[{\"ID\":\"user001@contoso.onmicrosoft.com\",\"Type\":5},{\"ID\":\"10037FFE91510806\",\"Type\":3}],\"TargetContextId\":\"41463f53-8812-40f4-890f-865bf6e35190\"}]"
| spath {} output=root
| stats count by root
| spath input=root Actor{} output=Actor
| mvexpand Actor
| spath input=Actor
| spath input=root Target{} output=Target
| mvexpand Target
| rename Type as Actor_Type
| spath input=Target
| spath input=root
| fields - Actor* root Target* count

https://docs.microsoft.com/en-Us/office/office-365-management-api/office-365-management-activity-api...
That's a lot of work.

If response is Array (not Object) like this, please make props.conf and LINE_BREAKER = (\[|,){\"CreationTime|(\]$)

0 Karma

adalbor
Builder

Thanks for the assistance. I tested that line_breaker config in our test environment but doesn't appear to be doing anything. It definitely is a lot of work to try and parse these logs. One example log I am looking at is over 32k characters!

0 Karma

to4kawa
Ultra Champion

Do you use SHOULD_LINEMERGE = true ?

0 Karma

to4kawa
Ultra Champion

can you provide sample logs link?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...