MSCS TA: How to ensure mscs:azure:eventhub sourcetype is mapping to CIM?

[sn0rlax]

Hi all. 

I'm trying to understand how to map my diagnostic setting AAD data coming in from an mscs:azure:eventhub sourcetype to CIM. 

I notice in the official docs for the TA, it mentions that the sourcetype above isn't mapped to CIM, however the azure:monitor:aad is mapped to CIM. 

I'm attempting to leverage Enterprise Security to build searches off of some UserRiskEvents data coming in, and would like to be able to reference datamodels.

So, is there any world I can take my existing data and transform it to match what's mapped to CIM? I envision like other TA's, that this can filter down to unique sourcetypes upon ingestion, while the Inputs on the IDM is set to a parent sourcetype. I can't confirm if that's true or not.

[deepakc · Accepted solution]

Once you configure your Azure event Hub inputs this should sourcetype mscs:azure:eventhub. Once the data comes in the Splunk TA will map to other sourcetypes, see below, these will then create the various CIM fields that can be mapped to the Alerts Data model (That's why you’re not seeing it being CIM compliant in the document list, as it’s a parent sourcetype)   

Note: Often Splunk TA's perform a lot of data props/transformations/Regex behind the scenes and CIM compliance work. 

The mscs:azure:eventhub sourcetype will point to the below sourcetypes and these are mapped to the Alerts Data model, now whether they contain the actual data you want is another matter, is this field you’re interested mapped to an Alerts Data model CIM field?  

mscs:azure:security:alert (CIM Mapped to Alerts Data model)

mscs:azure:security:recommendation (CIM Mapped to Alerts Data model)

The below sourcetype has many other data types, so various elements will map to the different datamodels.

azure:monitor:aad (Maps to Alerts/Authentication/Change)

So, in your case the Alerts Data model is most likely the main use case, so it’s best to get the data into an test index first, tune the Alerts Data model to point to the test index with the tag alert, this will kick the searches in for the Alert data model, you can then do some analysis on the CIM fields and see what you’re getting. 

If not seeing the fields you want, then your option is to use the Raw data for your searches, or you can create own data model and accelerate the data so it’s faster, but this is not CIM compliance in the true ES sense, it’s just making it faster and use of datamodels, which is fine, but maybe overkill. 

The general idea is to map as much as you can for CIM compliance or the ones recommended on the CIM Compliance page you never get it 100% 

Here are some Links for you to look at:

Alerts Data model - Look at your data and can you map them to a field or recommended ones, The TA should do most of this for you as its CIM compliant. 

https://docs.splunk.com/Documentation/CIM/5.0.2/User/Alerts

CIM validation - you can use this for analysis work

https://splunkbase.splunk.com/app/2968

MS Cloud TA - Info 

https://splunk.github.io/splunk-add-on-for-microsoft-cloud-services/

[sn0rlax]

A legend. Thank you for making that clear!