New install fails to start - Splunk Universal Forwarder

[nb41n]

I have successfully installed and configured Splunk and forwarders on OSX and Ubuntu systems but I have two Ubuntu systems where new installs fail to start and generate no error.

/opt/splunkforwarder$ sudo ./bin/splunk start 

....fails silently and appears not to write any error to logs. What is the process for debugging this?

Neil

[lukejadamec]

In the splunk/var/run/splunk folder look for a pid or lock file. If it is there, delete it and try to start splunk again.

[nb41n]

Thanks again. I don't believe Splunk is getting to the point of writing to any file as I don't get the licence screen that I would expect for an initial start. I certainly cannot find any output. It feels like a permissions problem but given I am starting Splunk as root that seems odd. Given Splunk is working on other similar servers locally I am convinced this is a local problem rather than a Splunk problem. I'll post the outcome when I can get it figured.

[lguinn2]

What log files do you get, if any? Is there a permissions problem that would prevent Splunk from writing/creating log files? If this is the case, I am truly surprised that Splunk does not give an error message...

Also, I would open a Support ticket at this point.
http://www.splunk.com/support

[nb41n]

Thanks for the response. I thought this had pointed me in the right direction as the log file mentioned in the docs did not exist. However, after 'touch'ing the log file and attempting to restart (both with and without --debug) still no joy.

[lguinn2]

Well, I have never had to do this - I've always found something in the logs 🙂

But you can start Splunk in debug mode

/opt/splunkforwarder$ sudo ./bin/splunk start --debug

More info in the docs here

I don't recommend this as a general debugging tool, but in this specific case it might help since you are getting no information at all. In "debug mode", I don't think Splunk will do much useful work!