Getting Data In

Limit output network bandwidth during peer decommissioning

edoardo_vicendo
Contributor

Hello,

We are in a multi-site indexer cluster environment, and we are going to upgrade our infrastructure from 3 Indexer to 6 Indexer.

Basically we will add 6 new indexers, and decommission the 3 old indexers.

Do you know which is the speed in MBytes/s that will be reached once the data will start to be copied from the 3 old Indexers to the 6 new indexers?

We did some test in our development environment, where we have simulated a similar scenario, and it seems it started copying very fast with 170 MBytes/s peak (we have checked it with an nmon session at the source Indexer machine). In order to start the copying process we had:

  • added the new Indexers to the Master Node
  • switched the Splunk HFs to forward data to the new Indexers
  • run, one by one on each old Indexer, the command splunk offline --enforce-counts

Before starting the test we have read the following documentation:

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Takeapeeroffline#Estimate_the_cluster_re...

In the official documentation it is mentioned:

10GB (rawdata and/or index files) from one peer to another across a LAN takes about 5-10 minutes

therefore copying speed should go from 136 Mbit/s to 272 Mbit/s.

If the speed we have observed is correct (more than 1Gbit/s), do you know if there is any way to limit the output bandwidth?

Thanks a lot,

Edoardo

Labels (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Check out the max_nonhot_rep_kBps setting in server.conf.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Check out the max_nonhot_rep_kBps setting in server.conf.

---
If this reply helps you, Karma would be appreciated.

edoardo_vicendo
Contributor

@richgalloway Thanks a lot for your suggestion!

Tested and it works perfectly, and you can even change it in the meanwhile:

splunk edit cluster-config -max_nonhot_rep_kBps <value>

 

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...