splunk crashing

[teknikKVU]

My instance of Splunk is crashing. In fact, it does not even start. After hitting "splunk start" at the command prompt, the last row is "Timed out waiting for splunkd to start."

Splunk version: 6.2.2
OS: Windows Server 2012R2

The splunkd logfile looks like this:

10-03-2018 10:48:01.687 +0200 INFO  loader - win-service: Starting as a Windows service: will run various system checks first...
10-03-2018 10:48:01.687 +0200 INFO  loader - win-service: Splunk starting as a local administrator
10-03-2018 10:48:01.687 +0200 INFO  loader - Automatic migration of modular inputs
10-03-2018 10:48:06.140 +0200 INFO  loader - win-service: Command pre-flight-checks ran successfully.
10-03-2018 10:48:08.505 +0200 INFO  loader - win-service: Command check-xml-files ran successfully.
10-03-2018 10:48:08.505 +0200 INFO  ServerConfig - My GUID is 73903130-C7B6-4F37-B105-DB98254D4590
10-03-2018 10:48:08.505 +0200 INFO  ServerConfig - My server name is "xxxx.local".
10-03-2018 10:48:08.505 +0200 INFO  ServerConfig - Found no site defined in server.conf
10-03-2018 10:48:08.505 +0200 INFO  ServerConfig - Found no hostname options in server.conf. Will attempt to use default for now.
10-03-2018 10:48:08.505 +0200 INFO  ServerConfig - Host name option is "".
10-03-2018 10:48:08.505 +0200 INFO  ServerConfig - My hostname is "XXXX".
10-03-2018 10:48:08.521 +0200 INFO  ServerConfig - Setting HTTP server compression state=on
10-03-2018 10:48:08.521 +0200 INFO  ServerConfig - Setting HTTP client compression state=0 (false)
10-03-2018 10:48:08.521 +0200 INFO  ServerConfig - Default output queue for file-based input: parsingQueue.
10-03-2018 10:48:08.521 +0200 INFO  loader - Splunkd starting (build 255606).
10-03-2018 10:48:08.521 +0200 INFO  loader - System info: Windows, XXXX, 2, 6, x64.
10-03-2018 10:48:08.521 +0200 INFO  loader - Detected 24 (virtual) CPUs, 12 CPU cores, and 16349MB RAM
10-03-2018 10:48:08.521 +0200 INFO  loader - Maximum number of threads (approximate): 8174
10-03-2018 10:48:08.521 +0200 INFO  loader - Arguments are: "-p" "8089"
10-03-2018 10:48:08.521 +0200 INFO  loader - Getting configuration data from: D:\Splunk\etc\myinstall\splunkd.xml
10-03-2018 10:48:08.521 +0200 INFO  loader - SPLUNK_MODULE_PATH environment variable not found - defaulting to D:\Splunk\etc\modules
10-03-2018 10:48:08.521 +0200 INFO  loader - loading modules from D:\Splunk\etc\modules
10-03-2018 10:48:08.521 +0200 INFO  loader - Writing out composite configuration file: D:\Splunk\var\run\splunk\composite.xml
10-03-2018 10:48:08.537 +0200 INFO  BundlesSetup - Setup stats for D:\Splunk\etc: wallclock_elapsed_msec=63, cpu_time_used=0.0625, shared_services_generation=1, shared_services_population=1

and the splunkd_utility:
10-03-2018 10:47:53.047 +0200 INFO  ServerConfig - My server name is "xxxxx.local".
10-03-2018 10:47:53.047 +0200 INFO  ServerConfig - Found no site defined in server.conf
10-03-2018 10:47:53.047 +0200 INFO  ServerConfig - Found no hostname options in server.conf. Will attempt to use default for now.
10-03-2018 10:47:53.047 +0200 INFO  ServerConfig - Host name option is "".
10-03-2018 10:47:53.047 +0200 INFO  ServerConfig - My hostname is "XXXX".
10-03-2018 10:47:53.062 +0200 INFO  ServerConfig - Setting HTTP server compression state=on
10-03-2018 10:47:53.062 +0200 INFO  ServerConfig - Setting HTTP client compression state=0 (false)
10-03-2018 10:47:53.062 +0200 INFO  ServerConfig - Default output queue for file-based input: parsingQueue.
10-03-2018 10:47:55.844 +0200 INFO  loader - Running utility: "validatedb"
10-03-2018 10:47:55.844 +0200 INFO  loader - Getting configuration data from: D:\Splunk\etc\myinstall\splunkd.xml
10-03-2018 10:47:55.844 +0200 INFO  loader - SPLUNK_MODULE_PATH environment variable not found - defaulting to D:\Splunk\etc\modules
10-03-2018 10:47:55.844 +0200 INFO  loader - loading modules from D:\Splunk\etc\modules
10-03-2018 10:47:55.844 +0200 INFO  loader - Writing out composite configuration file: D:\Splunk\var\run\splunk\composite.xml
10-03-2018 10:47:55.875 +0200 INFO  loader - Validated 19 indexes in 15.62 milliseconds
10-03-2018 10:47:56.234 +0200 INFO  ServerConfig - My server name is "xxxx.local".
10-03-2018 10:47:56.234 +0200 INFO  ServerConfig - Found no site defined in server.conf
10-03-2018 10:47:56.234 +0200 INFO  ServerConfig - Found no hostname options in server.conf. Will attempt to use default for now.
10-03-2018 10:47:56.234 +0200 INFO  ServerConfig - Host name option is "".
10-03-2018 10:47:56.234 +0200 INFO  ServerConfig - My hostname is "XXXX".
10-03-2018 10:47:56.234 +0200 INFO  ServerConfig - Setting HTTP server compression state=on
10-03-2018 10:47:56.234 +0200 INFO  ServerConfig - Setting HTTP client compression state=0 (false)
10-03-2018 10:47:56.234 +0200 INFO  ServerConfig - Default output queue for file-based input: parsingQueue.
10-03-2018 10:47:59.859 +0200 INFO  loader - Running utility: "check-transforms-keys"
10-03-2018 10:47:59.859 +0200 INFO  loader - Getting configuration data from: D:\Splunk\etc\myinstall\splunkd.xml
10-03-2018 10:47:59.859 +0200 INFO  loader - SPLUNK_MODULE_PATH environment variable not found - defaulting to D:\Splunk\etc\modules
10-03-2018 10:47:59.875 +0200 INFO  loader - loading modules from D:\Splunk\etc\modules
10-03-2018 10:47:59.875 +0200 INFO  loader - Writing out composite configuration file: D:\Splunk\var\run\splunk\composite.xml
10-03-2018 10:48:02.015 +0200 INFO  loader - Running utility: "validatedb"
10-03-2018 10:48:02.015 +0200 INFO  loader - Getting configuration data from: D:\Splunk\etc\myinstall\splunkd.xml
10-03-2018 10:48:02.015 +0200 INFO  loader - SPLUNK_MODULE_PATH environment variable not found - defaulting to D:\Splunk\etc\modules
10-03-2018 10:48:02.015 +0200 INFO  loader - loading modules from D:\Splunk\etc\modules
10-03-2018 10:48:02.031 +0200 INFO  loader - Writing out composite configuration file: D:\Splunk\var\run\splunk\composite.xml
10-03-2018 10:48:02.047 +0200 INFO  loader - Validated 19 indexes in 15.62 milliseconds
10-03-2018 10:48:02.390 +0200 INFO  ServerConfig - My server name is "xxxxx.local".
10-03-2018 10:48:02.390 +0200 INFO  ServerConfig - Found no site defined in server.conf
10-03-2018 10:48:02.390 +0200 INFO  ServerConfig - Found no hostname options in server.conf. Will attempt to use default for now.
10-03-2018 10:48:02.390 +0200 INFO  ServerConfig - Host name option is "".
10-03-2018 10:48:02.390 +0200 INFO  ServerConfig - My hostname is "XXXX".
10-03-2018 10:48:02.406 +0200 INFO  ServerConfig - Setting HTTP server compression state=on
10-03-2018 10:48:02.406 +0200 INFO  ServerConfig - Setting HTTP client compression state=0 (false)
10-03-2018 10:48:02.406 +0200 INFO  ServerConfig - Default output queue for file-based input: parsingQueue.
10-03-2018 10:48:05.703 +0200 INFO  loader - Running utility: "check-transforms-keys"
10-03-2018 10:48:05.703 +0200 INFO  loader - Getting configuration data from: D:\Splunk\etc\myinstall\splunkd.xml
10-03-2018 10:48:05.703 +0200 INFO  loader - SPLUNK_MODULE_PATH environment variable not found - defaulting to D:\Splunk\etc\modules
10-03-2018 10:48:05.703 +0200 INFO  loader - loading modules from D:\Splunk\etc\modules
10-03-2018 10:48:05.703 +0200 INFO  loader - Writing out composite configuration file: D:\Splunk\var\run\splunk\composite.xml

and a *.dmp file as well.

We have not changed the Splunk configuration. The operating system was shutting down the server due to overheating. But the server started up without any problem. And the filesystem seems to work as expected.
How do I handle this issue?

Thanks in advance.
//Magnus

[teknikKVU]

I can see *.dmp files in the ..\log\splunk directory. I´m aware of that the verison of splunk is outdated. I have inherited this splunk environment from a former colleague.

[ddrillic]

Do you see the crash logs under $SPLUNK_HOME/var/log/splunk? It's definitely the domain of Splunk Support.

[gjanders]

Perhaps it's time to upgrade to a version newer than 6.2.2, is that even supported on that Windows OS version?