Segmentation fault crash log on Indexer

[delgendy]

I have multiple crashes on my VM Linux servers "SUSE 12" that are running Splunk service in a cluster, mainly what is crashing are indexers and Search heads. We had different causes from the crash logs under Splunk which is Segmentation Fault and also on the var/log messages we see logs for crashes with a Segmentation fault. What can be monitored on the Server OS level to identify the root cause of the issue like what resources should be monitored to triage these crashes?

[codebuilder]

That means Splunk is consuming more memory than it is allowed by the OS, or it is exhausting all of the system memory and/or SWAP.

Ensure that ulimits are set correctly for your system:
https://docs.splunk.com/Documentation/Splunk/8.0.2/Installation/SystemRequirements#Considerations_re...

Increase the memory on your VM's or increase the size of your cluster, or decrease the workload.

[codebuilder]

If you found the suggestions to help, please "accept" the answer so that the community can benefit.