topic Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration? in Deployment Architecture

Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Wed, 21 Jan 2015 11:35:37 GMT

Hi All,

I am trying to setup Splunk in multisite environment. I have two sites in environment. Details of my environment are:

Master Node : 1
Search Head : 2 ( Each in site)
Peers : 4 ( Site 1 : 3 and Site 2 : 1)

With my current configuration i am getting error "Indexing not ready, fewer than replication factor peers are up." My current configuration is as follows:

MasterNode

[general]
site=site1

[clustering]

available_sites=site1,site2
mode = master
multisite=true
pass4SymmKey = abcde
site_replication_factor = origin:2,total:3
site_search_factor = origin:1,total:2

Please validate and let me know correct configurations to be done on Master and peers. With the error, I guess my replication_factor does not meet the criteria. Any help would be appreciated.

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

dxu_splunk — Mon, 28 Sep 2020 18:42:38 GMT

you'll need to have at least X peers per site, where X is the site_replication_factor's 'origin' number.

in your case here, you'll need 2 peers per site before indexing starts.

you can skip this requirement by setting indexing-ready on the master:

./splunk set indexing-ready

but you're replication factor will never be met (since site2 origin buckets can't possibly have 2 copies!)

btw, its also recommended that you set 'replication_factor' to match X as well (not the site one, the older replication_factor). see http://docs.splunk.com/Documentation/Splunk/6.1/Indexer/Migratetomultisite#How_the_cluster_migrates_and_maintains_single-site_buckets

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Thu, 22 Jan 2015 05:50:15 GMT

I want 1 replication copy of data at each site i.e one copy at site 1 and 1 copy at site 2 irrespective of data origin. Is that possible if i have 3 indexers at site 1 and 1 indexer at site 2 ? Please suggest configuration for the same?

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

dxu_splunk — Mon, 28 Sep 2020 18:43:11 GMT

sure. set site_replication_factor=origin:1,total:2

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

ppablo — Tue, 27 Jan 2015 04:39:29 GMT

Hi @Sourabhv06

If @dxu_splunk's answer and comment helped solve your question, don't forget to accept their response by clicking on "Accept" right below their answer to resolve this post. Thanks!

Patrick

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Mon, 28 Sep 2020 18:47:55 GMT

Hi dux_Splunk,

Once i had set up site_replication_factor as origin:1,total:2, i am not able to start the masterNode Service. Seems error in configuration. Could you please help me in setting up my multisite cluster based on the configuration mentioned above. Appreciate your quick help.

Thanks,
Sourabh

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

dxu_splunk — Thu, 29 Jan 2015 19:55:44 GMT

you'll need to change "replication_factor" as well to be less than the 'total' site replication factor.. lets set it to 1. (this affects only buckets that were created when multisite was turned off)

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Mon, 28 Sep 2020 18:48:32 GMT

Thanks!!! my replication factor and search factor are met after adding replication_factor and search_factor to my clustering stanza.

However i have a question. This search_factor and replication factor will be applicable for which bucktes ?

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

dxu_splunk — Mon, 28 Sep 2020 18:45:36 GMT

see http://docs.splunk.com/Documentation/Splunk/6.2.1/Indexer/Migratetomultisite#How_the_cluster_migrates_and_maintains_single-site_buckets

when u had multisite turned off, if there were buckets created during that time, they will follow replication_factor search_factor instead of site_replication_factor site_search_factor

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Tue, 03 Feb 2015 11:07:51 GMT

My multisite cluster is configured fine but my data is not getting replicated. Under indexes i can only see _audit and _internal. I am not able tyo see _main. Request you to help ASAP

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

dxu_splunk — Tue, 03 Feb 2015 20:38:41 GMT

did you set your index to be repFactor=auto?

http://docs.splunk.com/Documentation/Splunk/6.2.1/Indexer/Configurethepeerindexes

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Wed, 04 Feb 2015 13:51:44 GMT

Yes i had already distributed indexes.conf file with [main] stanza from master. I had mentioned repFactor = auto in that. But still no _main index is available.

Please suggest.

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

dxu_splunk — Wed, 04 Feb 2015 16:12:15 GMT

did u index data into main?

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Thu, 05 Feb 2015 10:41:39 GMT

Thanks For pointing it out. I was checking it before forwarding Data..

For testing I created a index test at one of my peer and edited a indexes.conf with following details

[test]
repFactor = auto

Once i restarted the peers, At master i can see the test index but now i am getting error as "Replication fator is not met and Search Factor is not met".

Please suggest where i have gone wrong. My server.conf is same as you stated.

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Thu, 05 Feb 2015 12:12:34 GMT

Moreover, while troubleshooting, I can see error for buckets as "cannot replicate as bucket is not servicable".

Appreciate your help!!!

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

dxu_splunk — Mon, 28 Sep 2020 18:51:42 GMT

whats ur rep factor, how many peers per site do u have? did you change the "replication_factor=1" and "search_factor=1"?

the "not serviceable" message goes away after a few minutes, dont worry too much about that and just check it again

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Mon, 28 Sep 2020 18:51:51 GMT

I have two sites. SIte 1 has 3 peers and site 2 has 1 peer. yes i had changed replication_factor = 1 and search_factor = 1 . My current config is

site_replication_factor = origin:1,site1:1,site2:1,total:2
site_search_factor = origin:1,site1:1,site2:1,total:2
replication_factor = 1
search_factor = 1

I worked fine till i created a custom index. After indexing data to custom index , error as stated above is persisting.

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Mon, 09 Feb 2015 05:50:37 GMT

Please suggest right config for my multisite cluster based on above information

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

dxu_splunk — Mon, 28 Sep 2020 18:52:42 GMT

can your peers communicate between each other? you'll need to make sure your peers' have their replication_port and mgmt_ports open to each other.

Re: Why am I getting error "Indexing not ready, fewer than replication factor peers are up" with my current Splunk 6.1 multisite cluster configuration?

Sourabhv05 — Tue, 10 Feb 2015 06:45:38 GMT

Yes i had enabled all ports and indexing under main index is working absolutely fine. Issue occurs when i create a custom index.

Once i create a custom index at one of my peer and than feed data to that index, at master i can see error as " replication factor not met " and " search factor not met" , though same config worked for main index