Splunk Search

o365:management:activity field extractions

adalbor
Builder

Has anyone had any success writing field extractions for O365 based events collected via the API?

The messages that are generated are HUGE and have multiple fields that contain multiple values.

I have tried to use eval and mvindex to see if its possible to extract those values but it doesn't appear to be working and I am wondering if its because of the JSON format.

Writing a regex for one of these events would have me ending up with something a page long lol.

Thanks,
Andrew

0 Karma

to4kawa
Ultra Champion
|makeresults
| eval _raw="[{\"CreationTime\":\"2015-06-29T20:03:19\",\"Id\":\"80c76bd2-9d81-4c57-a97a-accfc3443dca\",\"Operation\":\"PasswordLogonInitialAuthUsingPassword\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":9,\"ResultStatus\":\"failed\",\"UserKey\":\"1153977025279851686@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ClientIP\":\"134.170.188.221\",\"ObjectId\":\"admin@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"ExtendedProperties\":[{\"Name\":\"LoginError\",\"Value\":\"-2147217390;PP_E_BAD_PASSWORD;The entered and stored passwords do not match.\"}],\"Client\":\"Exchange\",\"LoginStatus\":-2147217390,\"UserDomain\":\"contoso.onmicrosoft.com\"},{\"CreationTime\":\"2015-06-29T20:03:34\",\"Id\":\"4e655d3f-35fa-42e0-b050-264b2d255c7a\",\"Operation\":\"PasswordLogonInitialAuthUsingPassword\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":9,\"ResultStatus\":\"success\",\"UserKey\":\"1153977025279851686@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ClientIP\":\"134.170.188.221\",\"ObjectId\":\"admin@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"Client\":\"Exchange\",\"LoginStatus\":0,\"UserDomain\":\"contoso.onmicrosoft.com\"},{\"CreationTime\":\"2015-06-29T20:04:55\",\"Id\":\"b567caf0-088e-4c1c-a4ea-633a1e3d66c8\",\"Operation\":\"Add User.\",\"OrganizationId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"RecordType\":8,\"ResultStatus\":\"success\",\"UserKey\":\"1003BFFD8EC47CA6@contoso.onmicrosoft.com\",\"UserType\":0,\"Workload\":\"AzureActiveDirectory\",\"ObjectId\":\"user001@contoso.onmicrosoft.com\",\"UserId\":\"admin@contoso.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":0,\"Actor\":[{\"ID\":\"1cef1fdb-ff52-48c4-8e4e-dfb5ea83d357\",\"Type\":2},{\"ID\":\"admin@contoso.onmicrosoft.com\",\"Type\":5},{\"ID\":\"1003BFFD8EC47CA6\",\"Type\":3}],\"ActorContextId\":\"41463f53-8812-40f4-890f-865bf6e35190\",\"InterSystemsId\":\"c2ced078-ad57-4079-a743-5c37f5284790\",\"IntraSystemId\":\"d1497f7e-15b4-49aa-83ad-11a17ca4a2f4\",\"Target\":[{\"ID\":\"user001@contoso.onmicrosoft.com\",\"Type\":5},{\"ID\":\"10037FFE91510806\",\"Type\":3}],\"TargetContextId\":\"41463f53-8812-40f4-890f-865bf6e35190\"}]"
| spath {} output=root
| stats count by root
| spath input=root Actor{} output=Actor
| mvexpand Actor
| spath input=Actor
| spath input=root Target{} output=Target
| mvexpand Target
| rename Type as Actor_Type
| spath input=Target
| spath input=root
| fields - Actor* root Target* count

https://docs.microsoft.com/en-Us/office/office-365-management-api/office-365-management-activity-api...
That's a lot of work.

If response is Array (not Object) like this, please make props.conf and LINE_BREAKER = (\[|,){\"CreationTime|(\]$)

0 Karma

adalbor
Builder

Thanks for the assistance. I tested that line_breaker config in our test environment but doesn't appear to be doing anything. It definitely is a lot of work to try and parse these logs. One example log I am looking at is over 32k characters!

0 Karma

to4kawa
Ultra Champion

Do you use SHOULD_LINEMERGE = true ?

0 Karma

to4kawa
Ultra Champion

can you provide sample logs link?

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer at Splunk .conf24 ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...