index replication unable to complete full index data

[kplem]

i have managed to set up cluster & index replication for 2 nodes for testing purpose. I have managed to replicate the index over to other splunk instance by editing the indexes.conf. However, the indexer seems to unable to replicate the full index. it only managed to index a very small amount of data over. Is there something missing. In short, i can see the replicated index on other splunk instance but the data is incomplete.

[jdunlea_splunk]

Also, remember that if you are setting up a second indexer to engage in replication with an existing indexer, then the existing data will NOT be replicated. You will only replicate data that was indexed AFTER you enabled replication.

[Steve_G_]

For more information on what happens to data that was already indexed prior to the indexer getting converted to a cluster peer (aka, "legacy data"), see http://docs.splunk.com/Documentation/Splunk/5.0.2/Indexer/Migratenon-clusteredindexerstoaclustereden...

[jdunlea_splunk]

When you add a new "index", (and the repFactor attribute is set to "auto" in indexes.conf) then all data that enters that index will be replicated. If you add a new "indexER" (Note difference between "index" and "indexer") then all data in that indexer will be replicated, if you have indeed set it up as an indexer in the cluster pool. Replication happens all the time for every 64bit chunks of data (as far as I know). Hope that helps.

[kplem]

I think that is what happen to my set up. it only replicates what the peer has after the cluster set up. In that case, is there a way to allow the new index to have full replication of the data of its peer? Also does the replication occurs all the time or a time can be scheduled?

[gkanapathy]

Why do you believe the data is incomplete? How are you looking? If you're simply looking at the file size of the buckets, it's entirely expected that a non-searchable replica will be considerably smaller (1/3 or less) than the size of a searchable replica. A searchable replica should be approximately (but not necessarily exactly) the same size on both systems.