Optimizing Accelerated Data Models

[danieladams]

My company is heavily using CIM accelerated data models for our security monitoring. We are currently experiencing performance issues and we think that data model acceleration is contributing to them. The searches that accelerate these data models are consistently the top memory-using searches, they run for a long time, and they are often behind. Is there a way to optimize these searches so that they aren't as taxing on the system? Here's how we have the CIM macros set up:

 (index="index1" AND sourcetype="sourcetype1")
    OR (index="index2" AND (sourcetype="sourcetype2" OR sourcetype="sourcetype3")
    OR (index="index3" AND sourcetype="sourcetype4")

And so forth. Any suggestions? We are on version 7.0.4.

[cstump_splunk]

First off, if you have not upgraded to either the latest core version of 7.2, 7.3, or 8.0, I recommend you do so as there have been updates that optimize the acceleration process.

Secondly, you should check to see if your datamodels have high cardinality fields, i.e fields with a high number of unique events. This leads to additional memory consumption on the Search Peers.
One way I like to do this is by using:

|datamodel <DM_name> acceleration_search 
| stats estdc(*) 
| transpose

Running this for a short 15-minute window in the past will tell you if any of the fields in the DM have high cardinality. If possible, either filter out some of the values in these fields or remove them for the DM definition.