Hi
We are collecting all logs from Windows (wineventlogs, windows, perfmon) from all the Domain Controllers. It's a huge amount of logs we are ingesting. What is the best practice to get the logs which has value for Splunk Enterprise Security and what logs are to be excluded?
The best practice for the collection of logs for use with Enterprise Security is to use (or build) an add-on that is compatible with the Splunk Common Information Model. One of the default filters on splunkbase is CIM compatibility. The Splunk Add-on for Microsoft Windows is CIM compatible, and will provide the eventtype and tagging requirements for the typical Windows Event Log sources. If you've already been ingesting Windows logs using that add-on, then you should have the log sources tagged properly for use with the CIM data models. Confirm that your Windows Event Log data is showing up in a datamodel (Example: review the Authentication data model using Pivot,) and if it's there you can move on to defining use-cases.
In a security environment, all data is relevant. So begin by defining the critical use-cases in your environment, and what data is required to fulfill those use-cases. Scope the data sources to CIM datamodels. To see which add-ons populate a given datamodel, use the 2 searches documented in this blog post. If you are working with correlation searches, review the search to see which datamodels are referenced.
With the list of required use-cases, and the knowledge of what data you need to fulfill those use-cases, you can choose to mark data sources that are less important to your current priorities and temporarily exclude them from collection.