We are having issus with Data models from Splunk_SA_CIM running for a very long time (hitting the limit) and causing out of memory (OOM) issues on our indexers. We have got brand new physical servers with 128 GB RAM and 48 Cores. The Enterprise security search head cluster has data models enabled which are both running on old and new hardware. Though we are getting OOM on new hardware and every run hits our 30+ min limit.
Example on configuration for auth DMA:
allow_old_summaries = true
allow_skew = 5%
backfill_time = -1d
cron_schedule = */5 * * * *
earliest_time = -6mon
hunk.compression_codec = -
hunk.dfs_block_size = 0
hunk.file_format = -
manual_rebuilds = true
max_concurrent = 1
max_time = 1800
Any tips on troubleshooting data models running for a very long time and causing out of memory (OOM)?
Thanks!
