Deployment Architecture

Can the replication factor be set per index?

Lowell
Super Champion

In a clustered configuration, is it possible to set different replication factors per index? This could come in handy for reducing the replication value for extremely large indexes or increasing the replication factor on super important indexes.

It seems like the docs are bit contradictory:

The Index.conf docs state:

repFactor = <nonnegative integer>|auto
    * Only relevant if this instance is a clustering slave (but see note about "auto" below).
    * See server.conf spec for details on clustering configuration.
    * Value of 0 turns off replication for this index.
    * If set to "auto", slave will use whatever value the master is configured with
    * Highest legal value is 4294967295

However, Configure the peer indexes page says:

The indexes.conf repFactor attribute
When you add a new index stanza, you must set the repFactor attribute to auto. This causes the index's data to be replicated to other peers in the cluster.

Is there some use for repFactor other than setting it to 0 or auto?

Tags (2)
1 Solution

svasan_splunk
Splunk Employee
Splunk Employee

The master has one replication_factor which it applies to all the indexes. The repFactor set per index currently only serves to turn on/off clustering (via auto and 0 resp) and cannot be used to set per-index replication factor. (So if clustering is turned on for that index via auto the master applies the same global replication policy to it.) So, yea, there is no per-index replication policy and the indexes.conf documentation should be clearer that the only values currently valid are 0 and auto.

View solution in original post

svasan_splunk
Splunk Employee
Splunk Employee

The master has one replication_factor which it applies to all the indexes. The repFactor set per index currently only serves to turn on/off clustering (via auto and 0 resp) and cannot be used to set per-index replication factor. (So if clustering is turned on for that index via auto the master applies the same global replication policy to it.) So, yea, there is no per-index replication policy and the indexes.conf documentation should be clearer that the only values currently valid are 0 and auto.

Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...