×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
Ask a Question
fgu
Loves-to-Learn Lots
Member since: ‎11-13-2020
‎05-12-2022
Community Statistics
Posts 2
Solutions 0
Karma Given 0
Karma Received 0
Member Since ‎11-13-2020
View All

Re: indexes.conf recommendation for large volume o...

by in Getting Data In
‎11-16-2020 07:59 AM
‎11-16-2020 07:59 AM
Thank you guys for taking the time to share your insight! that is helpful.   The data is coming from 1 source. This is why it is in a single index. It is using also the same type of storage (local SSD), that is why I was considering keeping as many warm bucket as I can. About the volume of data per indexer, I am much (much!) higher that what you recommend.  I thought it was Ok cause of the spec of the server I am using: (48c/96T cpu @2 .3 Ghz, 128GB of RAM, 5TB local SSD in RAID10). Would you recommend to  keep index volume low even with this type of configuration? Is there a rule of thumb I can follow to find out what the right volume of data per day based on HW specs?   thanks /Fabien ... View more

indexes.conf recommendation for large volume of da...

by in Getting Data In
‎11-13-2020 06:33 PM
‎11-13-2020 06:33 PM
Hi, I am looking for any recommendation when  dealing with such scenario. On one instance or one indexer, 300-400GB of data per day in a single index. Is there any recommended configuration for such index? So far, I came with few changes: -  increase maxTotalDataSizeMB to go beyond the 500GB default and meet my requirement in term of retention. - Enable maxDataSize = auto_high_volume (host bucket to 10GB) What I am considering is to increase the number of hot bucket, cause with 6 Hot bucket by default, it is only 60GB of data which is not even 24 Hours of data. Should I increase it ? or should I only increase the number of warm bucket? or both? Are warm buckets also 10GB each? If my disk capacity and performance allows it, can I keep only warm bucket for my max retention (30 days) and dont use Cold? Any advice or feedbacks with this type of scenario? thanks /Fabien ... View more
Labels
Contact Me
Online Status
Offline
Date Last Visited
‎05-12-2022 05:44 PM