Questions about Data Model new source addition.

[zacksoft_wf]

I have  this 'Email' Data Model in ES. The model is populated by macro and tags(2 eventypes populated by saved searches)
(`cim_Email_indexes`) tag=IS_Email 
The two eventtypes have IS_Email tag associated to them . Now,  A new source needs to be fed into the dataModel. The fields of the new source  are cim compatible but are not fed into the dataModel. And I checked the corresponding eventType and there were some tags associated to it but IS_Email tag wasn't there. So, To add the data from this new EventType into the datamodel, if I just add IS_Email tag into it(the eventtype), is it sufficient ? Or anything else is required ? If this is sufficient, then after adding the Tag, do I need to rebuild the Email DataModel  ?

[zacksoft_wf]

In my instance I  see all the eventtypes tagged to IS_Email are also tagged with 'email'.  
Also I checked the TA sourcetypes and its fields are parsed as per the cim complaint fields.  
In that case just adding the 'email'  and 'Is_Email' tag to the new eventtype is enough to fed its data to the datamodel ?

[gcusello]

Hi @zacksoft_wf,

what technology are you ingesting?

what's the Add-On you're using?

as I said, if you're using a CIM 4.x compliance Add-On you don't have do do nothing, otherwise you have to check CIM 4.x compliance of your data source, you can use the Apps I listed in my previous answer.

Add the tag could not be sufficient.

Ciao.

Giuseppe

[zacksoft_wf]

ingesting ProofPoint TA data
proof point email security

[gcusello]

Hi @zacksoft_wf,

I suppose, you're speaking of Proofpoint Email Security Add-On, is it correct?

This TA is CIM 4.x compliant, so it should correctly run.

Ciao.

Giuseppe

[zacksoft_wf]

Yes.
Thank you so much for the explanation.

[gcusello]

Hi @zacksoft_wf,

good for you, see next time!

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉

[gcusello · Accepted solution]

Hi @zacksoft_wf,

at first, you have to check if the new source you're ingesting is CIM 4.x compliant.

If it's CIM 4.x compliant you don't have to do nothing, if it isn't you have to normalize your TA to make your source compliant.

In other words, it isn't suffient to add the tag to the eventtype, also because your tag "IS_mail" isn't CIM compliant, the correct tag is "mail".

The first hint is to search in apps.splunk.com an Add-On CIM 4.x compliant for your data source, so you don't have to do nothing, otherwise you have to use an app as CIM Validator (https://splunkbase.splunk.com/app/2968/) or Splunk Common Information Model (CIM) App (https://splunkbase.splunk.com/app/1621/) and manually make all the normalizations (field names, field values, tags, etc...).

Ciao.

Giuseppe

[zacksoft_wf]

If I may just ask  a related question,
What if I ever decide to stop the feed from one eventtype. Will just by removing the 'email' tag from the corresponding eventtype do the job ? And no re-build or anything required ?

[gcusello]

Hi @zacksoft_wf,

for new questions, I hint to open a different question so more people can help you better and quicker than me!

Anyway, if you remove a tag from an eventtype, new data from that data source will not be indexed in the Data Model, but already indexed data remain in it, if you want to delete them from the Data Model, you have to rebuild the Data Model.

If you don't want to modify the TA, you could also modify the rule in the Data Model.

Why to do this?

Ciao.

Giuseppe