Getting Data In

Recomended total index/disk size for one indexer

Motivator

Is there a recommended/optimum size of all the indexes that one indexer can handle?

I have not analysed this, but indexers that have 500gb of indexed data restart faster that those with 2tb. I am wondering if anyone has run into any problems because of this. Or if others have rules like "whenever our indexers manage more than xy gb we add another indexer".

Thanks

Chris

Tags (2)
1 Solution

Influencer

chris,

Have you checked out Splunk's doc on Capacity Planning? - docs here, this informs you of the best practices on scaling Splunk to meet your needs.

Also, if you have not already done so, I would check Splunk's docs on Retirement and Archiving... docs here, as this may help with performance on larger indexers where you do not require as much data to be kept "fresh" (or searchable) in the warm buckets.

Apologies if I have missed the point of your question. But generally I would think an indexer with more "active" data would take longer to restart than an instance with less.

Regards,

MHibbin

View solution in original post

Influencer

chris,

Have you checked out Splunk's doc on Capacity Planning? - docs here, this informs you of the best practices on scaling Splunk to meet your needs.

Also, if you have not already done so, I would check Splunk's docs on Retirement and Archiving... docs here, as this may help with performance on larger indexers where you do not require as much data to be kept "fresh" (or searchable) in the warm buckets.

Apologies if I have missed the point of your question. But generally I would think an indexer with more "active" data would take longer to restart than an instance with less.

Regards,

MHibbin

View solution in original post

Influencer

sorry to avoid giving a straight answer, it just is one of those things that depends on soo many factors, it would be hard to predict over that length of time.

0 Karma

Motivator

Yeah I guess you're right. I'll check with our support. I just thought that maybe someone might have come across issues with the amount of data/disk space they have on their indexers.

0 Karma

Influencer

the amount of searches performed. Splunk will have to search all the data (obviously it does this in a effecient manner, but still), so you will be multiplying the work load for each search used. If you are an Enterprise customer, and require some dedicated help with your setup, you could always ask support (they will have a lot more experience with many different set-ups).

Influencer

I suppose it really depends on your setup. There would be no right or wrong answer. I suppose based on the default of 500GB/Index, if I had more than this I would have a new indexer and then use a distributed search across the multiple boxes (depending on available funds). You would also have to take into consideration that if you TB's of data on one machine, what happens if you have a complete failure on that machine... you'd lose everything, so it would be better to have some backup/distribution (like how you RAID disk to cope with faults). Also you have to take into condiseration...

0 Karma

Motivator

Thanks for the answer, I've just realized, that my question is probably to generic. Correct me if I am wrong but there does not seem to be a recommendation for the maximum amount of disk space one indexer can handle (If I have to keep my data for 10 years and I want it to be searchable (stupid requirement I know) I will end up with huge disk space. Up to how much data can one Splunk indexer handle if the HW corresponds to the requirements in the Capacity Planning section of the docs)

0 Karma