Enterprise Security: how does the TA determine that a certain index/event-set is cim compliant?

[danielbb]

How does the TA determine that a certain index/event-set is cim compliant? Does it require all the fields to match or only a certain sub-set.

[diogofgm]

TA's contain, among other things, field extractions. TA's that try to be compliant with the CIM tend to use certain fields naming conventions in order to, regardless the tech/vendor, data that represents the same info has the same name (e.g. instead of "username" or "user_name" we use "user", instead of "d_ip" or "dstip" we use dest_ip). There is a big list of fields in the CIM. Most TAs that state to be "CIM compliant" usually use this naming convention while handling field extractions/alias.

ES relies heavily on data models built on top of this CIM naming convention. Each data model expects to find certain fields in your tagged events. you can find more about what tags and CIM fields each datamodel expects here in the docs.

If some important field is not being correctly extracted from your raw data (e.g user from authentication data) or tagged, the data models won't pick it up and some panels in ES might not show data.

Check this presentation from conf on data normalisation using the CIM:
https://conf.splunk.com/files/2017/slides/the-power-of-data-normalization-a-look-at-cim-under-the-ho...

[danielbb]

Great. Let me try to clarify myself.

When I run | datamodel Web search | table * it shows the table with all the fields. Now Web.action is populated while Web.app is empty. Therefore, I assume that Web.app is an optional field for the Web cim/datamodel. Is it right? so, how do I know which fields are optional and which ones are mandatory?

[jawaharas · Accepted solution]

AFAIK, TA don't validate an index for CIM compliant. It has to be done manually by the user. In most cases, only key fields (which are CIM compliant) are needed for a Splunk App to work properly.

CIM Validator is a great Splunk app for CIM validation.

[danielbb]

The screenshots at SA-cim_vladiator are really impressive.

[danielbb]

The CIM Validator seems like a great app - thank you.

[jawaharas]

I'm glad it helped.

[jawaharas]

@danielbb

Please accept the answer if it significantly helped resolve your query for the benefit of other forum members, who might run into a similar issue.

[danielbb]

Sure thing.

You said - In most cases, only key fields (which are CIM compliant) are needed for a TA to work properly.

How can I find out which ones are needed?

[jawaharas]

During indexing or search time , the fields are extracted by Addons (as per CIM complaint if configured properly) and the fields are used by Splunk Apps/Dashboards/Datamodels.

Splunk Enterprise Security Suite app utilizes bunch of data models as mentioned here. The list of fields used by each datamodel also provided.

[danielbb]

Makes perfect sense, but which fields are needed in order to certify a certain event as cim compliant?

[jawaharas]

It's depend on the use case and app that you use. You can get the list of required fields either from the Splunk query used in the dashboard/reports/datamodels or from the app's documentation.