Caution on Retention – Impact of Accelerated Data Model and Report acceleration when using volume based retention policies

[sat94541]

Be careful when you set size-based retention limits for your indexes so they do not take up too much disk storage space. By default, report acceleration summaries can theoretically take up an unlimited amount of disk space. This can be a problem if you're also locking down the maximum data size of your indexes or index volumes.

[rbal_splunk · Accepted solution]

The good news is that you can optionally configure retention limits for your report acceleration summaries or Data Model acceleration.

Note: Although report acceleration summaries are unbounded in size by default, they are tied to raw data in your warm and hot index buckets and will age along with it. When events pass out of the hot/warm buckets into cold buckets, they are likewise removed from the related summaries. Same is true for Data Model Acceleration.

For example, by default, report acceleration summaries live alongside the hot and warm buckets in your index at homePath/../summary/. In other words, if in indexes.conf the homePath for the hot and warm buckets in your index is:

homePath = /opt/splunk/var/lib/splunk/index1/db
Then summaries that map to buckets in that index will be created at:
homePath/opt/splunk/var/lib/splunk/index1/summary

for example you can have index like
[winevents]
coldPath= volume:Seconday/winevents/colddb
homePath= volume:primary/winevents/db
tstatsHomePath= volume:_Data_Model\$_index_name\datamodel_summary
summaryHomePath== volume:_reportacc_summaries\$_index_name\datamodel_summary

So to manage disk utilization better, you will need to define separate volume for

homepath>hot and warm bucket
coldPath>Cold Buckets
Data Model Acceleration
Report Acceleration

And set each of the volume like
[volume:primary]
path =

maxVolumeDataSizeMB= < allowed size>

[volume:seconday]
path =

maxVolumeDataSizeMB= < allowed size>

[volume:_ Data_Model]
path = $SPLUNK_DB
maxVolumeDataSizeMB= < allowed size>

[volume: :_reportacc_summaries]
path = /Data/report_acceleration/
maxVolumeDataSizeMB= < allowed size>

This information is documented at location --http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Manageacceleratedsearchsummaries and look link http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Acceleratedatamodels Section “Configure size-based retention for data model summaries”