Splunk Replication a Good BCP Alternative?

andrewkenth — Wed, 16 Oct 2013 19:33:36 GMT

We have been working with our architects to determine the most appropriate solution for BCP. Does Splunk replicate non processed information such as dashboards, users etc.? Is it a viable solution for running a BCP instance of Splunk?

Re: Splunk Replication a Good BCP Alternative?

dwaddle — Thu, 17 Oct 2013 17:15:35 GMT

Splunk clustering / index replication does not replicate any search head configuration or knowledge objects like views, lookup tables, etc. It only replicates the actual indexed data across peer indexers.

As of current version (6.0) there is still a SPOF in the cluster master - you would need to find some way to make it highly available as well.

You could deal with the search head data using Search Head pooling, but would need a way to make the NFS highly available. You might be able to use rsync or something like it to keep a secondary search head configured identically without doing pooling.

Another gotcha is that clustering has no concept of locality / affinity / topology. That is, if you have 4 "primary" indexers and 4 "DR" indexers then you need a minimum replication factor of 5 to be sure there is at least one copy of each bucket at the DR site.

You will also need substantial bandwidth with very low latency between the primary site and the DR site.

You would need something like (indexer_throughput * number_of_indexers * (replicationfactor - 1)) bandwith available for Splunk. In the 8 indexer example above with an RF=5, each of 4 indexers at the primary site will be replicating 4 copies of their data to other indexers. In a worst-case scenario (from a bandwidth perspective), all 4 replicated copies will be to the DR site. So, if each indexer is processing 500 KBytes/sec of data to be indexed - then 4 indexers, each sending 4 copies of their 500 KBytes/sec data across the WAN to the DR site, will result in 8,000KBytes/sec of total WAN bandwidth needed.

So there are definitely caveats but it is viable. I would strongly suggest discussing your planned deployment with a Splunk Architect or Professional Services.

topic Splunk Replication a Good BCP Alternative? in Deployment Architecture

Splunk Replication a Good BCP Alternative?

Re: Splunk Replication a Good BCP Alternative?