How to make a CIM compliant data in Splunk ES?

[AL3Z]

Hi, 

I have a existing dlp data model, Can we add the indexed dlp data to exisiting one to make a cim compliant OR
we need to create a new datamodel  to add the data ?

[AL3Z]

 I want to check the netscope addon in props.conf and transforms.conf all the respective fields are available in the logs or not so that I can map them. But i am on splunk cloud how do I look for the app props and transform conf files from GUI ?
Thanks..

[gcusello]

Hi @AL3Z,

you can install in Splunk Cloud (or in your on-premise test instance) one of this apps:

SA-cim_vladiator (https://splunkbase.splunk.com/app/2968) I used it many times!

CIM_Buddy (https://splunkbase.splunk.com/app/6259) I used sometimes.

CIM Toolkit for Splunk Supporting Common Information Model Add-on (https://splunkbase.splunk.com/app/6243) I neved used it, but I'll try it.

Ciao.

Giuseppe

[AL3Z]

Hi,
@gcusello 
@richgalloway 
Even after dlp data is added to the dlp datamodel, I didnt see any dlp incidents in the datamodel, what could be the problem here ?

Thanks

[gcusello]

in the screenshot you shared there's the Data Model's structure, you don't see any data.

To see Data Model data you have to use it in a search

| datamodel DLP_Incidents

If you haven't data, run the constraints search to see id you have correctly tagged data.

Ciao.

Giuseppe

[AL3Z]

In fact this is the correct search

 
| datamodel DLP DLP_Incidents

I can see only the 1 event no fields 😞

[richgalloway]

One event is a start.  It means the event is tagged correctly.

The next step is install one or more TAs to extract fields from the data.  Then ensure the data uses CIM field (if the TAs do not do that for you).

[AL3Z]

@richgalloway 
Could you please guide me how we could  install one or more TAs to extract fields from the data.  ?

[richgalloway]

Search splunkbase (apps.splunk.com) for a TA/add-on that is pertinent to your data.  Follow the instructions there to install the TA.

For general instructions about installing add-ons, see https://docs.splunk.com/Documentation/AddOns/released/Overview/Installingadd-ons

[AL3Z]

@richgalloway 

We already have a netskope addon for our data.

[richgalloway]

If you still aren't getting fields extracted from your data then either you have the wrong TA or you need an additional TA.  You may need to create your own TA if one does not exist in splunkbase.

Have you confirmed the sourcetype in your inputs.conf file matches the sourcetype in props.conf (in a TA or created by you) and also matches the sourcetype expected by the apps you are using?

[AL3Z]

We are using the splunk cloud we don't have access to the inputs and props conf

Could you please clarify  me TA is same as add-on right?

And one more thing i can Tstats to some fields of the datamodel inspite of getting one event when I search DLP datamodel.

Thanks 

 

[richgalloway]

Splunk Cloud users do not have file system access, but they still have access to config files through the apps they install.  They also can use REST calls and the GUI to examine many settings.

Yes, "TA" and "add-on" mean the same thing.  A TA is a Technical Add-on and is a term used to describe an app that help onboard data, typically by extracting fields.

I'm not even going to try to read those tiny screenshots.  If the tstats command can find fields then they must be extracted.  Try searching in Verbose Mode to see if they show up in your other searches.

[AL3Z]

Where do we get to see the rest calls and gui settings in the splunk cloud and using stats i cant see some fields of the DM.

[richgalloway]

Go to Settings->Source types and select your sourcetype.  Click on the Advanced tab and you'll see the props.conf settings for that sourcetype.

To see them using REST, try this 

| rest /services/configs/conf-props | search eai:acl.app=<<your app>>

[AL3Z]

How we can edit these props if need changes? Do we have feasibility in splunk cloud ?

Thanks

[gcusello]

Hi @AL3Z,

yes, you have it also in Splunk Cloud at [Settings > Sourcetypes].

Ciao.

Giuseppe

[AL3Z]

Using this search I can see all the fields, In this dm there is no incident_id field, how we need to get in this dm, do we need to write regex for that if yes where we need to mention the regex in ?

[gcusello]

Hi @AL3Z,

if a field isn't present in the Data Model, you can customize it adding the field and rebuilding the DM.

About how to extract the field, it depends on the field itself and how you extract it in a normal search (regex, calculated field, alias etc...).

Ciao.

Giuseppe

[AL3Z]

@gcusello ,

If I tstats the dm it is showing  an Error in 'DataModelEvaluator': Data model 'DLP_Incidents' was not found.
what we need to do on this ?

[gcusello]

Hi @AL3Z,

sorry, I always have problems to identify the Data Model name, please try this:

| datamodel DLP  DLP_Incidents

Ciao.

Giuseppe