Deployment Architecture

Configurations for a "baseline" + "updates" type index

cplunk
Explorer

Hi,

I'm trying to figure out the most recommended way to set up an index that stores data ingested in the following manner:

1) Every ~30 days a baseline of events is sent, specifying the current "truth".

2) Between baselines, small updates are ingested, specifying diffs from the previous baseline.

A baseline would be around ~1 GB, and the small updates would be ~1 MB every few days.

Queries on this index will build a "current state" by querying the baseline + the updates since. This would require a baseline + updates to be kept in warm buckets. 

I was wondering what would be the best indexes.conf configuration for this case?

My initial thought was:

frozenTimePeriodInSecs=7776000  # 90 days to keep ~3 baselines
maxDataSize=2000 # max size of a baseline
maxWarmDBCount=30

The reason I set maxWarmDBCount to 30 was in case of an update every day, and automatic rolling from hot to warm bucket. If hot buckets can stay hot for multiple days, I could reduce this number.

Any inputs?

Thanks!

 

 

0 Karma
Get Updates on the Splunk Community!

.conf25 Community Recap

Hello Splunkers, And just like that, .conf25 is in the books! What an incredible few days — full of learning, ...

Splunk App Developers | .conf25 Recap & What’s Next

If you stopped by the Builder Bar at .conf25 this year, thank you! The retro tech beer garden vibes were ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...