Getting Data In

Limit output network bandwidth during peer decommissioning

edoardo_vicendo
Builder

Hello,

We are in a multi-site indexer cluster environment, and we are going to upgrade our infrastructure from 3 Indexer to 6 Indexer.

Basically we will add 6 new indexers, and decommission the 3 old indexers.

Do you know which is the speed in MBytes/s that will be reached once the data will start to be copied from the 3 old Indexers to the 6 new indexers?

We did some test in our development environment, where we have simulated a similar scenario, and it seems it started copying very fast with 170 MBytes/s peak (we have checked it with an nmon session at the source Indexer machine). In order to start the copying process we had:

  • added the new Indexers to the Master Node
  • switched the Splunk HFs to forward data to the new Indexers
  • run, one by one on each old Indexer, the command splunk offline --enforce-counts

Before starting the test we have read the following documentation:

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Takeapeeroffline#Estimate_the_cluster_re...

In the official documentation it is mentioned:

10GB (rawdata and/or index files) from one peer to another across a LAN takes about 5-10 minutes

therefore copying speed should go from 136 Mbit/s to 272 Mbit/s.

If the speed we have observed is correct (more than 1Gbit/s), do you know if there is any way to limit the output bandwidth?

Thanks a lot,

Edoardo

Labels (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Check out the max_nonhot_rep_kBps setting in server.conf.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Check out the max_nonhot_rep_kBps setting in server.conf.

---
If this reply helps you, Karma would be appreciated.

edoardo_vicendo
Builder

@richgalloway Thanks a lot for your suggestion!

Tested and it works perfectly, and you can even change it in the meanwhile:

splunk edit cluster-config -max_nonhot_rep_kBps <value>

 

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...