Deployment Server scalability best practices.

[hrawat]

How to improve Splunk Deployment server scalability?

[hrawat · Accepted solution]

Deployment Server Performance  

To improve DS scalability instead of increasing phoneHomeIntervalInSecs to hours. Following are must have configs.

On the DS server.conf

[sslConfig]
sslServerSessionTimeout = 7200
[httpServer]
dedicatedIoThreads = 10

Increase net.core.somaxconn, If less than 512.

sysctl -a |grep net.core.somaxconn

Then have splunk side config set to 512.

In $SPLUNK_HOME/etc/splunk-launch.conf on DS.
SPLUNK_LISTEN_BACKLOG = 512

Make sure net.core.somaxconn >=SPLUNK_LISTEN_BACKLOG

On DC side  server.conf

useSslClientSessionCache=true

Upgrade all DCs to 7.1.3 and above to have higher timeouts. Before 7.1.3 these are hardcoded 5 sec.
In deploymentclient.conf

connect_timeout = <positive integer>
* Default: 60

send_timeout = <positive integer>
* Default: 60

recv_timeout = <positive integer>
* Default: 60