Help with Archiving Frozen data

[kiran331]

Hi,

We have multi site indexer clustering with 2 sites, 3 indexers in each site with RF-3 and SF-2. Each indexer has its only drive for archives both rb_ and db_ buckets. Is there a way to archive together on a single standalone Indexer without archiving duplicate data? Please advise the best practice to achieve it.

[adonio]

@kiran331,
the challenge is described well in docs:
http://docs.splunk.com/Documentation/Splunk/7.0.1/Indexer/Automatearchiving#Data_archiving_and_index...
not sure if it answers you needs but here is a trick. lets assume you either have another indexer (doesnt matter in which site) or, you can take one indexer out (again doesnt matter which site) and create a third "cluster" with 1 indexer call it site 3
now you can setup replication to make sure that this single indexer receives a copy of all the data in server.conf
read here:
https://docs.splunk.com/Documentation/Splunk/7.0.1/Indexer/Sitereplicationfactor#Examples
now, you have an archiving site consists of one indexer... single point of failure, but you can easily carve another "site" and set the replication rules to have a copy on "archiving site1" and "archiving site2"

hope it helps

[knalla]

Thanks adonio, currently all the data is going to site1 and replicating 1 copy to site 2. I think single indexer with site3 will be best option for us, but it if i move all frozen db_ buckets from site1 to a disk, will db_* buckets have replicated data too?

server.conf:
site_replication_factor = origin:2,site1:1,site2:1,total:3
site_search_factor = origin:1,site1:1,site2:1,total:2

[adonio]

keep buckets where they belong, they can only be restored at the indexer that they were created.
my suggestion above, is for moving forward.
your server.conf looks would have also site3=1 (archive indexer) for replication_factor that will ensure that you have a replicated copy in that indexer.
once applied, verify you have all replicas in site3 and modify you archiving procedure to freeze data only from site3. other data can age out safely as you supposed to have a replica and a frozen copy on site 3.
please let us know how it worked for you

[kiran331]

Thanks Adonio, I have a question, will the all archives from a site with 1 replicated copy with 3 indexers and archives from 1 indexer site with 1 replicated be same (in terms of disk space)?

[adonio]

@kiran331,
hope i understand your question correctly.
when you "force" a copy on a site that has 1 indexer it means that regardless of which site the original copy is "landed" on, it will replicate a copy to that site (with single indexer).
setting an archiving (freezing) policy on this indexer will assure that you will have exactly 1 copy of all data on this indexers frozen data path.
2 challenges here are:
a. you will be able to thaw (recover) data only to this indexer.
b. you will probably need more storage on this indexer to accommodate retention.

hope it answers it