Adding a Source type to a Data Model

[fedayn05]

Hello,

I hope you are doing well.

I did integrate my firewall fortigate to Splunk using udp syslog , what i did exactly is that I created a new source type and associated it with FortigateSplunkApp. 

All was good until now that I am configuring Splunk ES , the issue is that correlation searches use the Network-Traffic Data model and my source type is not part of that Data Model i get no notables.

The question is there a way i can ass my source type to be part of the Network_Traffic Data Model

Thank you,

[PickleRick]

I would add a bit of info to @richgalloway 's response.

Firstly - yes, 100% on that "don't modify CIM datamodels". There are some border cases when I've seen people do that but editing a Splunk-provided datamodel definition will only cause you pain. Next time a new revision of CIM app comes out you'll have to review the changes, manually merge them into your own edited datamodel... it's practically never worth the effort. So keep the DM definition as it is.

And second - CIM datamodels are meant as standard to which your data should conform, not the other way around. So if you have a good CIM-compliant addon for parsing your data, it should be properly adjusting your fields so that they fit the datamodel. If not, it's up to you to not only extract raw fields from your data but also adjust them (sometimes calculate new values from existing ones, sometimes simply rename fields so they fit the datamodel...) to be CIM-compliant.

And then you should configure your sourcetype (again - you modify Splunk's behaviour regarding your data, not the CIM definition!) so that it "marks" certain events with proper tags as per the datmodels' definitions.

There is an app - add-on builder which can help you (but it also has some flaws and some people dislike it heavily) do that.

So I understand that for now you're at the beginning of this journey.

[fedayn05]

Hello,

I did add those tags to my fortigate index , and now all events are tagged with the following tags : network, communication and also a tag called error.

But I still do not get any notables. I do not know if that error tag is the issue or not.

Thank you

[richgalloway]

Make sure the Network Traffic DM is scanning the index that contains the Fortigate data (there's a macro for it).

Also, make sure the Fortigate events are CIM-compliant.  If the data doesn't use CIM fields it won't appear in the DM.

Run the data model's constraint manually in a search (in the ES app) to confirm the events are found and the proper fields are extracted.

[fedayn05]

Hello,

Thank you for your response, As I am new to this , can you please help me or provide me with a guide to go through this.

I would really appreciate that.

Thank you again.

[richgalloway]

I recommend using the SA-cim_vladiator (https://splunkbase.splunk.com/app/2968) app to confirm your data is CIM-compliant.  Add EVAL and FIELDALIAS settings to props.conf to create CIM-compliant fields as needed. 

[richgalloway]

Don't use sourcetypes to integrate data into a data model and DON"T modify a data model.

Instead, use tags to integrate the data into the DM.  The DM will search specified indexes for events with specific tags.  In the case of the Network Traffic model, the tags are 'network' and 'communicate'.  Ensure the Fortigate data has those tags and it will become part of the DM.