How to migrate buckets from a standalone indexer to a multisite cluster environment?

[sylim_splunk]

I had an older standalone splunk indexer. I set up a new multisite cluster (2 indexers, site rep/search factor of 2) and have all data available at both sites. Splunk version is the latest, 7.1.2.1.
I want to take the old data (legacy indexer, not replicated), and have it replicate in the new multisite cluster.

I am dealing with ~1tb of logs across many years. Just adding the old indexer to my search heads is not a valid workaround -- 100% data availability at both sites is my primary concern.

• Steps followed - 1.Set up an indexer (single_idx) with index named "test". 2.Set up an indexer cluster (multi_idx1, multi_idx_2) with index "test". 3.Copy a warm bucket from single_idx to multi_idx_1 4.Rename the bucket to append the multi_idx_1 guid following the clustered bucket naming convention. 5.Watch the multi_idx_2 db folder and watch for a rb_ folder corresponding to the manually added bucket. It never replicates to the other site indexer.

[oliverj]

With the release of 7.2, I see this in the new server.conf:

constrain_singlesite_buckets =
* Only valid for mode=master and is only used if multisite is true.
* Specifies whether the cluster keeps single-site buckets within one site
in multisite clustering.
* When this setting is "true", buckets in a single site cluster do not
replicate outside of their site. The buckets follow 'replication_factor'
'search factor' policies rather than 'site_replication_factor'
'site_search_factor' policies. This is to mimic the behavior of
single-site clustering.
* When this setting is "false", buckets in non-multisite clusters can
replicate across sites, and must meet the specified
'site_replication_factor' and 'site_search_factor' policies.
* Default: true

[oliverj]

Update: This worked.
Added "constrain_singlesite_buckets = false" to the server.conf on my cluster master.

[sylim_splunk]

Thanks for trying it out.

[sylim_splunk · Accepted solution]

As of this writing where the latest version is 7.1.2.1, this is not a supported feature.
The steps you have done works for the case of the migration from standalone to single site cluster. But it is not working for the case where you migrate to multisite cluster as the buckets created in single site cluster/standalone don't have site information.
This will be implemented and available in 7.2+, where a parameter, "constrain_singlesite_buckets" is available in server.conf.

[mibrown_splunk]

Also worth a comment that pre-7.2, you could still migrate the buckets from standalone to multisite cluster. The difference pre-7.2 is that the buckets will replicate but only within the site the buckets were sent to, based on the single-site replication settings. Like you said, due to the new parameter in 7.2, you get to replicate buckets across the multisite environment as though they were generated natively.