One thing I have notice, is when I try to resync one of these 3 jobs in pending, in the drop down menu to choose where to resync the bucket, I only have indexers of the origin's site of the bucket, and only one indexer from the other site. So I can't force réplication to another indexer in the other site... The "View Becket Details" in the details of the task in pending give me for the bucket this info : ---- Replication count by site : site 1:1 site 2:7 Search count by site site 1:1 --- Is there a way to force replication on a desired indexer on the other side ? (because Splunk list me only one and always the same indexer on the other site) Thanks ! ... View more

Hi, | rest /services/cluster/manager/buckets | where multisite_bucket=0 AND standalone=0 From MC gives me the same error messages. Port 8089 have been tested between MC > 8089 > Indexer(s) and is open. I think it is because the web service is off on my inderxers... But no time to dig for this actually. Priority is to get back SF / RP to normal green. One thing I have notice, is when I try to resync one of these 3 jobs in pending, in the drop down menu to choose where to resync the bucket, I only have indexers of the origin's site of the bucket, and only one indexer from the other site. So I can't force réplication to another indexer in the other site... The check of the task in pending give me for the bucket this info : ---- Replication count by site : site 1:1 site 2:7 Search count by site site 1:1 --- Is there a way to force replication on a desired indexer on the other side ? (because Splunk list me only one and always the same indexer on the other site) Thanks ! ... View more

Thanks for all your help and advices.  I will try the rest command on the MC as u suggest tomorow, im back home now. Normally MC is well configured. I will update after searching. But I agree with you, I think I will have to open a case @splunk 😞 Best regards ... View more

Modification have been made on the CM in etc/system/local/server.conf Added in etc/system/local/server.conf (+restart of the CM) : "contrain_singlesite_buckets = false" No change. Performed another rolling restart. No change Still have job in pending who are impossible to resyncronize. Any suggestion in order to find where comes from the problem ? Many thanks ... View more

Thanks for the return. The rest command is not working for me... I have errors message in the CM search on all my peers indexers like : [Peer1,Peer2,....] HTTP status not OK, Code=503, Service Unavailable The web part is stopped on all my indexers... any other way to have this info ? Thanks ... View more

Thanks, I was reading the same page ^^ I keep u updated. I just want to verify before pushing the modificiation in the CM server.conf (only) + restart CM deamon : [clustering] mode = manager constrain_singlesite_buckets = false Do you know how to perform : To see how many buckets will require conversion to multisite, use services/cluster/manager/buckets?filter=multisite_bucket=false&filter=standalone=false before changing the manager node configuration. Thanks Thanks ... View more

Hi and thanks for the support. >>> If your cluster has been recently migrated from single site to multisite  : NO >>> Restarting CM _might_ resolve your issue : Already done (CM only and rolling restart with no effect for RP/SF) The operation was to add new indexers, and then decomission the old ones. It's a multisite since the build of everything. But I have "constrain_singlesite_buckets=true", on the CM and on the INDX in etc/system/default/server.conf Maybe it was buckets from the beginning of my infrastructure, at a time that the multi site cluster was not builded and operational ? Do you know the impact of changing constrain_singlesite_buckets to false ? Many thanks ... View more

Command passed under search of my Monitoring Console, I have all my 17 Indexers "Up" with the right site repartion. I dont see the decomissionned indexer who dont have any splunkd running. (Splunkd have been disabled). Thanks ... View more

  splunk btool server list clustering | grep factor   Hi thanks, here is the output : etc/system/default/servers.conf >>> ack_factor = 0 etc/apps/MULTI_SITE_APP/local/server.conf >>> replication factor = 2 etc/apps/MULTI_SITE_APP/local/server.conf >>>search_factor = 2 etc/apps/MULTI_SITE_APP/local/server.conf >>>site_replication_factor = origin:1, site1:1, site2:1 total 2 etc/apps/MULTI_SITE_APP/local/server.conf >>>site_search_factor = origin:1, site1:1, site2:1 total 2 etc/system/default/server.conf >>> replication factor = 3 Regards, ... View more

Hi and thanks for the reply. And what are your site RF/SF > can you be more spécific please ? In the server.conf in my CM ? (I will check that when back to work tomorow. For the sites details : 2 site with 18 indexers on each side. So 9 on one site and 8 + 1 decommissioned on the other site.  I get back to you tomorow morning. Regards, ... View more

Hi all, Im under Splunk Version 9.0.2. After decomissionning one indexer in a multi site clustering, I cant retrieve my SF / RP. Rolling restart and restart CM have no effect. Got 3 SF tasks in pending with the same message as mentionned in this post.  Id like to be clean before decomissionning another indexer.  Any help will be appreciate in order to retrieve my SF/RP Thanks ... View more

》An other intorragation, is it normal to only have default DataModels visible (and not all my Datamodels) from CM (Settings/DataModels)  ? My DM are ok.... sorry for that ... View more

Thanks for your return, You are right. The decomissionned indexer is now on state "Graceful shutdown" and buckets count is 0. Took 2.5 days to decomission 20 To of datas.  But SF / RF is still not green. 3 SF tasks are still in pending, i tried to resync thems but no change.  Should I now do a rolling restart after removed my decomissionned indexer in order to get back my SF / RP ?  Or simply restart my CM splunk deamon ? An other intorragation, is it normal to only have default DataModels visible (and not all my Datamodels) from CM (Settings/DataModels)  ? Many thanks    ... View more

Hi and many thank for your fast return. About my doubts of my results, I think i wasnt sure that the counting were correct, beacause I have more than 20 000 assets... The usage of the "stats" command is for now, a bit foggy to me. So i wasnt sure of my search request. While trying differents things, the differents results was not concordants... so I wanted to check that my method was correct. Many thanks again for the solution, it do perfectly the result I needed ! Best regards my new Splunk friend ^^   ... View more