Data trim

[Siddharthnegi]

I have some questions regarding data trim

[VatsalJagani]

@Siddharthnegi - Please ask your question along with proper details so Splunk Community can help you.

Community Moderator,

Vatsal Jagani

[Siddharthnegi]

I want to ask from which version of splunk data trim feature has been added.

[gcusello]

Hi @Siddharthnegi,

please see my answer to your previous question: https://community.splunk.com/t5/Splunk-Search/Searching-Cisco-ASA-sourcetype-in-Splunk/m-p/653115#M2...

Ciao.

Giuseppe

[VatsalJagani]

@Siddharthnegi - What do you mean by the splunk data trim? Can you describe it? Where do you it, in query or somewhere else?

[Siddharthnegi]

yes,

so you allocate some storage to index and a retention period. now if that storage fills up before the retention period is over splunk intelligence starts to trim old data so that new data can be stored.

lets say you have allocated 500gb to particular index for 6 months retention time. And everyday 70gb data is being produced and stored in that index, now in less than 10 days storage will be filled so splunk intelligence then trim cold data from cold bucket.

[gcusello]

Hi @Siddharthnegi,

this means that you have to redesign your storage dimension because ingesting 70 GB/day and wanting to retain them for 180 days, you need:

Index_size = daily_ingestion * reduction_factor * retention_days

in other words:

Index_size = 70 * 0.5 * 180 = 6300 GB

you can also divide your storage:

• Hot and Warm Buckets in an high performance storage (at least 800 IOPS)
• Cold buckets in a less performant and expensive storage

Ciao.

Giuseppe