How to change ulimit and THP in Splunk after ver 7.2.2

[yutaka1005]

OS : Amazon Linux 2
Splunk Ver : 7.2.3

In Splunk before ver 7.2.2, there is start script in /etc/init.d, so I was adding commands to the file like below.

 splunk_start() {
   echo Starting Splunk...
   ulimit -n 64000
   ulimit -u 16000
   if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
    echo never > /sys/kernel/mm/transparent_hugepage/enabled
   fi
   if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
    echo never > /sys/kernel/mm/transparent_hugepage/defrag
   fi

   if test -f /sys/kernel/mm/redhat_transparent_hugepage/enabled; then
    echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
   fi
   if test -f /sys/kernel/mm/redhat_transparent_hugepage/defrag; then
    echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
   fi

But in Splunk after ver 7.2.2, if Splunk detect systemd, it makes unit files to run splunkd service, and there is not start script in /etc/init.d.

So I added options ExecStartPre,LimitNOFILE,LimitNPROC to unit file /etc/systemd/system/Splunkd.service, and I could change ulimit and disable THP.

ExecStartPre=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"
ExecStart=/opt/splunk/bin/splunk _internal_launch_under_systemd
LimitNOFILE=64000
LimitNPROC=16000

*Answers I referred
https://answers.splunk.com/answers/590209/rhelcentos-7-systemd-not-honoring-ulimits.html

Anyway, I want to know whether this way is best, OR is there another best practice reccomended by Splunk?

[yutaka1005 · Accepted solution]

Splunk Support team said this method is no problem.

[inventsekar]

Hi @yutaka1005 , your above ideas/methods looks perfect.

more info on these topics:
https://answers.splunk.com/answers/188875/how-do-i-disable-transparent-huge-pages-thp-and-co.html

https://docs.splunk.com/Documentation/Splunk/7.2.2/Admin/RunSplunkassystemdservice

[droehr_splunk]

very good idea... thx

[yutaka1005]

Thank you for comment!

I asked Splunk Support team, and they said this method is no problem!