Is there a way to specify an alternate location for a Splunk forwarder to do its Splunk crash dumps?
Yes, this is a function of the OS. In solaris, you use coreadm to instruct the system where to place core dump files. In the few versions of linux that I know anything about, it's a sysctl. You'll add, say, kernel.core_pattern=/var/core/core_%h_%e_%u_%g_%t_%p
to /etc/sysctl.conf and reboot.
No. There is no way to tell splunk where to place the crash*.log files. You can control the location of some other log files from: $SPLUNK_HOME/etc/log.cfg More at: http://www.splunk.com/base/Documentation/latest/Admin/Splunklogfiles
Yes, this is a function of the OS. In solaris, you use coreadm to instruct the system where to place core dump files. In the few versions of linux that I know anything about, it's a sysctl. You'll add, say, kernel.core_pattern=/var/core/core_%h_%e_%u_%g_%t_%p
to /etc/sysctl.conf and reboot.
I asked Splunk tech support this very question. They told me that Splunk will dump its core wherever the operating system specifies.
One recommendation someone had was to write a wrapper around the splunk binary. On our AIX system, it looks like this:
splunk.sh:
#!/bin/ksh
ulimit -c 0 #disable core dumps
exec /splunk/bin/splunk $1
The ulimit -c 0 option tells the operating system not to make a core dump. (Technically, it creates a core dump of 0 bytes in size, but it never seems to make an actual core file).
So when you start Splunk, you'd do: /splunk/bin/splunk.sh start
(Note: because of the $1, this only works for commands that have one parameter.)
The other option was to rename the splunk binary to splunk_real, then call the above shell script "splunk". That option didn't sit well with me, however.
Hope that helps!
Nice tip, thanks!
You can have your script pass all parameters by just changing $1
to $*