Splunk instance crashed with incomplete crash file (crash.log*) under $SPLUNK_HOME/splunk/var/log/splunk/.
The crash log file is seems truncated with below error and so, it's not useful for troubleshooting.
Crash log has overflowed the allowed limit (512kB), truncated
How to increase crash log file size? Any help is welcome one. Thanks.
Let me answer to my own question with the inputs from Splunk support.
512kb
limit for crash log is hardcoded in Splunk crash dump creation code. It is a pretty reasonable size and captures the necessary information.
Some crashes, like ones from a search process, may contain the last lines from the search.log and could get cut off. The setting itself is NOT configurable unfortunately.
Let me answer to my own question with the inputs from Splunk support.
512kb
limit for crash log is hardcoded in Splunk crash dump creation code. It is a pretty reasonable size and captures the necessary information.
Some crashes, like ones from a search process, may contain the last lines from the search.log and could get cut off. The setting itself is NOT configurable unfortunately.
For some of the log inputs you can change the default settings in the $SPLUNK_HOME/etc/log.cfg file. Although for the crash log I havent found anything yet.
Have you tried contacting Splunk support on this issue?
I haven't. I will raise a Splunk support case and update here if I found solution. Cheers.