Hello All,
I have tried updating ulimits values but it is not persistant in all the instances. Production environment which I am working on is having cluster/licensce master, search head cluster, indexer cluster and few heavy forwarder instances.
- As per splunk recommendation tried updating ulimits in /etc/security/limits.conf file and done the service restart. Post that ulimits got persistently updated only in indexer instances and in rest of the instances there was no changes in ulimits value.
- Tried hard coding ulimits values in etc/init.d/splunk file inside splunk start () fucntion as well when it boot starts and rebooted the instances. Post that ulimits persistently updated in only indexer and search head instances
In heavy forwarder and licence master instances even if I try to increasing ulimits after couple of hpours it is getting reduced and can you please suggest ehat can be done to set the ulimits persistantly
-
Have you tried setting ulimits when starting Splunk? I put this code in my init.d/splunk file.
change_ulimit() {
ulimit -Hn 65535
ulimit -Sn 65535
ulimit -Hu 20480
ulimit -Su 20480
ulimit -Hf unlimited
ulimit -Sf unlimited
}
Then invoke the function when starting Splunk
case "$1" in
start)
change_ulimit
splunk_start
;;
stop)
splunk_stop
;;
restart)
change_ulimit
splunk_restart
;;
status)
splunk_status
;;
esac
If you use systemd, try these settings in the Splunk service file:
LimitNOFILE = 65535
LimitNPROC = 20480
LimitFSIZE = infinity
And if your environment is using selinux then remember run restoreconn after changes!
r. Ismo
updated:
Create file /etc/security/limits.d/21-splunk.conf with content
splunk soft nofile 65536
splunk hard nofile 65536
Add also other user specific limits to this file if/when needed.
Then run:
restorecon -FvvR /etc/security/limits.d
After that new login for user splunk and then it should works.
r. Ismo
And it requires reboot of the server also.
Actual not, jus logout and login for that user which limit you have changed.
r. Ismo
/etc/init.d/splunk is executed at server booting. So I don’t think it works the way you think.
You could run it by hand any time you want. No need to wait a reboot.