Recommended RF and SF in Indexer Clustering

[rselv21]

Hi Everyone,

Can you please suggest the recommended RF and SF number for Splunk clustered environment with total 9 indexers and 7 search heads in Indexer Clustering set-up.  

And also, please let me know the concept regarding how many copies will replicated in each indexers out of 9?

[isoutamo]

I have explain this and also searchable buckets in this post https://community.splunk.com/t5/Deployment-Architecture/SF-and-RF-How-much-count-should-we-keep/m-p/....

You didn't say if you have

• normal cluster in one site or multisite cluster
• have you a smart store in use

If You have smart store then you must use SF=RF, otherwise it's possible that spunk try to upload your new bucket from replicate bucket which is not searchable and then it didn't work with S2.

If you have multisite cluster then you will have also site replication factor and site search factor which manages  those copies over your sites.

As @PickleRick already said your installation arise questions as you said that you have 9 indexers and 7 search head. I need to say that it's quite weird combination especially if those SHs are individual and not in SHC.

[livehybrid]

Hi @rselv21 

The defaults for Replication Factor (RF) and Search Factor (SF) are 3 and 2 respectively, this essentially allows for 1 node to be out of service without a loss of functionality. If a secondary node also becomes unavailable then some buckets may not have any searchable copies available to search. Although non-searchable copies of the buckets can be made searchable, doing so takes time

As others have said, its basically a trade off between risk of data loss (RF), risk of lowered availability (SF) and storage availability (Cost). 

The rule of thumb for sizing is that a searchable replica bucket uses approx 35% of the original raw ingest size, and a non-searchable bucket uses approx 15% of the raw ingest size, so a RF:SF of 3:2 uses approx 85% of the raw ingested volume per day. 

Personally, the defaults are pretty good for the majority, its only when you're limited by resource, OR have a large amount of searches running across a much larger indexer cluster that I would adjust these, as distributing more copies across a larger cluster can improve performance by ensuring there are no hotspots of data.

Check out the following docs for some good background on this: https://docs.splunk.com/Documentation/Splunk/9.4.2/Indexer/Thesearchfactor

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[PickleRick]

RF and SF are site-specific parameters depending on the organization risk tolerance and cost constraints. I've seen sites with RF=SF=1 (yes, you get no data resilience but management of some aspects of your environment is easier) as well as SF=RF=number of indexers-1 (you shouldn't set your RF to be equal to your number of indexers!).

BTW, 9 indexers and 7 SHs? Are those standalone SHs? Or do you have a 7-member SH cluster? (which seems an overkill unless you have a very specific use case)

[kiran_panchavat · Accepted solution]

@rselv21 

As @richgalloway  said, These values are generally recommended because they provide a good balance between data availability and storage efficiency. 

SF and RF - How much count should we keep ? - Splunk Community 

Solved: Search factor vs Replication factor-If I change my... - Splunk Community

In a clustered environment, the Replication Factor (RF) determines how many copies of each data bucket are maintained across the indexers. For example, with an RF of 3, each data bucket will have 3 copies spread across different indexers. This ensures that if one or two indexers fail, the data is still available on the remaining indexers. 

The Search Factor (SF) specifies how many of these replicated copies are searchable. With an SF of 2, two of the replicated copies will be in a searchable state, allowing search heads to query the data even if one of the searchable copies becomes unavailable 

With 9 indexers and an RF of 3, each data bucket will be replicated across 3 of the 9 indexers. This means that any given bucket will have 3 copies, ensuring redundancy and high availability.

[PickleRick]

"With 9 indexers and an RF of 3, each data bucket will be replicated across 3 of the 9 indexers. This means that any given bucket will have 3 copies, ensuring redundancy and high availability."

This is not entrely true.

1. "Ensuring" redundancy and HA depends on organization's risk acceptance level.

But more importantly

2. "each data bucket will be replicated across 3 indexers" suggests that whole bucket data will be replicated which is not true, as you pointed out earlier mentioning RF and SF.

[richgalloway]

This is a question only you can answer based on your risk tolerance level and how much storage you have.

I recommend RF/SF values of at least 2.  Higher values offer more protection against failure, but at the cost of additional storage.