topic Re: Questions about Data Model new source addition. in Splunk Enterprise Security

Questions about Data Model new source addition.

zacksoft_wf — Wed, 23 Feb 2022 18:42:15 GMT

I have this 'Email' Data Model in ES. The model is populated by macro and tags(2 eventypes populated by saved searches)
(`cim_Email_indexes`) tag=IS_Email
The two eventtypes have IS_Email tag associated to them . Now, A new source needs to be fed into the dataModel. The fields of the new source are cim compatible but are not fed into the dataModel. And I checked the corresponding eventType and there were some tags associated to it but IS_Email tag wasn't there. So, To add the data from this new EventType into the datamodel, if I just add IS_Email tag into it(the eventtype), is it sufficient ? Or anything else is required ? If this is sufficient, then after adding the Tag, do I need to rebuild the Email DataModel ?

Re: Questions about Data Model new source addition.

gcusello — Wed, 23 Feb 2022 09:17:20 GMT

Hi @zacksoft_wf,

at first, you have to check if the new source you're ingesting is CIM 4.x compliant.

If it's CIM 4.x compliant you don't have to do nothing, if it isn't you have to normalize your TA to make your source compliant.

In other words, it isn't suffient to add the tag to the eventtype, also because your tag "IS_mail" isn't CIM compliant, the correct tag is "mail".

The first hint is to search in apps.splunk.com an Add-On CIM 4.x compliant for your data source, so you don't have to do nothing, otherwise you have to use an app as CIM Validator (https://splunkbase.splunk.com/app/2968/) or Splunk Common Information Model (CIM) App (https://splunkbase.splunk.com/app/1621/) and manually make all the normalizations (field names, field values, tags, etc...).

Ciao.

Giuseppe

Re: Questions about Data Model new source addition.

zacksoft_wf — Wed, 23 Feb 2022 10:16:42 GMT

In my instance I see all the eventtypes tagged to IS_Email are also tagged with 'email'.
Also I checked the TA sourcetypes and its fields are parsed as per the cim complaint fields.
In that case just adding the 'email' and 'Is_Email' tag to the new eventtype is enough to fed its data to the datamodel ?

Re: Questions about Data Model new source addition.

gcusello — Wed, 23 Feb 2022 10:20:56 GMT

Hi @zacksoft_wf,

what technology are you ingesting?

what's the Add-On you're using?

as I said, if you're using a CIM 4.x compliance Add-On you don't have do do nothing, otherwise you have to check CIM 4.x compliance of your data source, you can use the Apps I listed in my previous answer.

Add the tag could not be sufficient.

Ciao.

Giuseppe

Re: Questions about Data Model new source addition.

zacksoft_wf — Wed, 23 Feb 2022 11:22:12 GMT

ingesting ProofPoint TA data
proof point email security

Re: Questions about Data Model new source addition.

gcusello — Wed, 23 Feb 2022 11:16:28 GMT

Hi @zacksoft_wf,

I suppose, you're speaking of Proofpoint Email Security Add-On, is it correct?

This TA is CIM 4.x compliant, so it should correctly run.

Ciao.

Giuseppe

Re: Questions about Data Model new source addition.

zacksoft_wf — Wed, 23 Feb 2022 11:24:47 GMT

Yes.
Thank you so much for the explanation.

Re: Questions about Data Model new source addition.

gcusello — Wed, 23 Feb 2022 11:26:44 GMT

Hi @zacksoft_wf,

good for you, see next time!

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉

Re: Questions about Data Model new source addition.

zacksoft_wf — Wed, 23 Feb 2022 11:29:50 GMT

If I may just ask a related question,
What if I ever decide to stop the feed from one eventtype. Will just by removing the 'email' tag from the corresponding eventtype do the job ? And no re-build or anything required ?

Re: Questions about Data Model new source addition.

gcusello — Wed, 23 Feb 2022 11:45:16 GMT

Hi @zacksoft_wf,

for new questions, I hint to open a different question so more people can help you better and quicker than me!

Anyway, if you remove a tag from an eventtype, new data from that data source will not be indexed in the Data Model, but already indexed data remain in it, if you want to delete them from the Data Model, you have to rebuild the Data Model.

If you don't want to modify the TA, you could also modify the rule in the Data Model.

Why to do this?

Ciao.

Giuseppe