Deployment Architecture

SF and RF never met while adding an indexer to multi-site cluster

mchang_splunk
Splunk Employee
Splunk Employee

We migrate single site to multisite cluster by following the document:
https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Migratetomultisite

However, SF and RF never met:
Missing enough suitable candidates to create searchable copy in order to meet replication policy. Missing={ site2:1 }
Missing enough suitable candidates to create replicated copy in order to meet replication policy. Missing={ site1:1 }

There is only 1 bucket listed in fixup task, the bucket name is already multi-site bucket name, but still can't meet SF and RF with above message.

Tried:
1. verified GUID is correct as peer and restart everything, not working
2. remove all files in the bucket, only leave /rawdata/journal.gz, then do ./splunk rebuild , not working

What else I can do to resolve this issue?

0 Karma
1 Solution

mchang_splunk
Splunk Employee
Splunk Employee

The root cause is the bucket is marked as single-site ("origin_site" = default):

cat /db_1525916743_1521375071_1_C89A6E4E-B848-4326-8755-1D614DFA4F9A/.bucketManifest

id,path,"raw_size","event_count","host_count","source_count","sourcetype_count","size_on_disk",modtime,"frozen_in_cluster","origin_site","tsidx_minified","journal_size"

"unix~1~C89A6E4E-B848-4326-8755-1D614DFA4F9A","db_1525916743_1521375071_1_C89A6E4E-B848-4326-8755-1D614DFA4F9A",1104884368,5732042,6,21,5,425193472,1545273340,0,default,0,92858030 

change default to site1 should resolve the issue

id,path,"raw_size","event_count","host_count","source_count","sourcetype_count","size_on_disk",modtime,"frozen_in_cluster","origin_site","tsidx_minified","journal_size"

"unix~1~C89A6E4E-B848-4326-8755-1D614DFA4F9A","db_1525916743_1521375071_1_C89A6E4E-B848-4326-8755-1D614DFA4F9A",1104884368,5732042,6,21,5,425193472,1545273340,0,site1,0,92858030 

View solution in original post

boz_8058
Explorer

This is because the bucket has come from a single instance.

To get this to automatically replicate, append the following in the clustering stanza in $SPLUNK_HOME/splunk/etc/system/local/server.conf on the index cluster master;

[clustering]
constrain_singlesite_buckets = false

 

0 Karma

mchang_splunk
Splunk Employee
Splunk Employee

The root cause is the bucket is marked as single-site ("origin_site" = default):

cat /db_1525916743_1521375071_1_C89A6E4E-B848-4326-8755-1D614DFA4F9A/.bucketManifest

id,path,"raw_size","event_count","host_count","source_count","sourcetype_count","size_on_disk",modtime,"frozen_in_cluster","origin_site","tsidx_minified","journal_size"

"unix~1~C89A6E4E-B848-4326-8755-1D614DFA4F9A","db_1525916743_1521375071_1_C89A6E4E-B848-4326-8755-1D614DFA4F9A",1104884368,5732042,6,21,5,425193472,1545273340,0,default,0,92858030 

change default to site1 should resolve the issue

id,path,"raw_size","event_count","host_count","source_count","sourcetype_count","size_on_disk",modtime,"frozen_in_cluster","origin_site","tsidx_minified","journal_size"

"unix~1~C89A6E4E-B848-4326-8755-1D614DFA4F9A","db_1525916743_1521375071_1_C89A6E4E-B848-4326-8755-1D614DFA4F9A",1104884368,5732042,6,21,5,425193472,1545273340,0,site1,0,92858030 
Get Updates on the Splunk Community!

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...

Combine Multiline Logs into a Single Event with SOCK: a Step-by-Step Guide for ...

Combine multiline logs into a single event with SOCK - a step-by-step guide for newbies Olga Malita The ...

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...