Running CIM in our ES instance and I had some questions around tagging or NOT tagging data.
Whats the best way to go about excluding certain events from being tagged by a data model.
Example: Malware data model looks at everything in the CIM indexes you specify during setup.
Our AV reports on events that aren't malware related but because the data model references that index everything is tagged as malware thus resulting in false positives. Whats the best practice/way to go about ensuring that data is not tagged as such?
Can we exclude by signature? Or event code?
Re-onboard your data such as the bad AV data. Break its link to the tags until you fix it.
Limiting the macros on the data models is important for performance but the above is the key drivers.
The index alone does not dictate what appears in the Malware datamodel. The data is being tagged somewhere with tag=malware. Splunk tags your data based on eventtypes. Somewhere you likely have an eventtype that is specifying that ALL of your AV data should be tagged as "malware".
You can use the
| rest command to see where your malware tags are coming from and find the culprit:
| rest splunk_server=local servicesNS/-/-/saved/eventtypes | search tags=malware | table eai:acl.app, eai:acl.sharing eai:acl.perms.read, title, search, tags, author
Here you should be able to find the eventtype query (labeled "search" in this table) that is including unwanted data.
So just to verify I am doing the right thing.
After validating that app that is adding the tags, I go into eventtypes.conf in that app and place in local folder then remove tag from that specific eventtype?
Once you find the app that has the eventtype / tag combo configured, you can disable the tag with a local/tags.conf file. Likely in default/tags.conf for that app there is an entry like this:
[eventtype_name_here] malware = enabled
To disable, you can create a corresponding local/tags.conf entry
[eventtype_name_here] malware = disabled
Then you can go about creating eventtype and tag combo that works for your data.
Thanks, great info and appreciate the help.
Here is the odd thing, when I look at the apps applying the tags. None of the searches actually line up with these events being tagged with the malware tag. These are McAfee AV events and the McAfee TA eventtypes.conf does not appear to tag these specific events with the malware tag.
Do not be fooled by the name
cim_malware_indexes; you can and SHOULD specify as much as you can in there, including
sourcetypes. That is the best way to limit your datamodels.
Thanks for the info. Exactly what I am doing now.
I have run into an interesting issue maybe someone can help with. In creating the search in the data model constraints there is a discrepancy.
A search in the preview pane for the Data model for past 7 days returns 2 results in ES.
When I run it on my SHC it returns 30 results using the same query.
I have tried without the macro as the macro is not set on the SHC.
Any ideas why it would do this?
CIM searches cannot see your
Knowledge Objects unless you create them inside of its app (DO NOT DO THIS) or you change your
All Apps AKA
Global. Also be aware that if you are using
Enterprise Security, that there is an application whitelist that may be factoring in, too:
In the newest version of ES, this feature has been removed.