Why can't I get my Splunk Enterprise installation to work on Ubuntu?

[louiseaxon]

Hi,

I'm trying to install Splunk Enterprise on a Virtualbox VM running Ubuntu 16.04. I get the following error after starting Splunk (by running dpkg on the .deb download) for the first time and going through the licensing info:

Splunk> 4TW

Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
ERROR: pid 2132 terminated with signal 9
Checking kvstore port [8191]: open
Checking configuration... Done.
Checking critical directories... Done
ERROR: pid 2145 terminated with signal 9
Validating databases (splunkd validatedb) failed with code '-1'. If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue

I have tried all the advice I could find online for this error:
- my user is added to the splunk group
- added line OPTIMISTIC_ABOUT_FILE_LOCKING = 1 in $SPLUNK_HOME/etc/splunk-launch.conf
- set $SPLUNK_HOME through line SPLUNK_HOME = "/opt/splunk" in /etc/environment

Does anyone have any advice on what else to try, or if any of the above doesn't look right?

Thanks in advance

[kamlesh_vaghela]

HI @louiseaxon,

I have faced same issue with mac.

https://answers.splunk.com/answers/614068/issue-with-splunk-in-mac-machine.html

but with respect of my answer I found a reply for ubuntu also.

https://answers.splunk.com/answers/306998/why-am-i-getting-homepathoptsplunkvarlibsplunkaudi.html

Can you please try that solution?

Add this line to $SPLUNK_HOME/etc/splunk-launch.conf

OPTIMISTIC_ABOUT_FILE_LOCKING = 1

Please read all comments and discussion of provided link. This will help you to understand more.

Thanks
Kamlesh

[jrodmantcell]

The exit code of -1 means this isn't the standard "unsupported filesystem" problem. Exit code of -1 is a bug of course, because negative exit codes are undefined, but the main point is if it's not 1, it's something else went wrong.

[dimarra]

I am experiencing the same issue after upgrading from ubuntu 17.04 to 17.10.

I then upgraded from splunk 6.6.3 to 6.6.5 hoping that this is resolve in this patch. IT IS NOT.

Did a fresh install of splunk 6.6.5, issues is still NOT RESOLVED.

Splunk> Now with more code!

Checking prerequisites...
        Checking http port [8000]: open
        Checking mgmt port [8089]: open
        Checking appserver port [127.0.0.1:8065]: open
ERROR: pid 19180 terminated with signal 9
        Checking kvstore port [8191]: open
        Checking configuration...  Done.
                Creating: /opt/splunk/var/lib/splunk
                Creating: /opt/splunk/var/run/splunk
                Creating: /opt/splunk/var/run/splunk/appserver/i18n
                Creating: /opt/splunk/var/run/splunk/appserver/modules/static/css
                Creating: /opt/splunk/var/run/splunk/upload
                Creating: /opt/splunk/var/spool/splunk
                Creating: /opt/splunk/var/spool/dirmoncache
                Creating: /opt/splunk/var/lib/splunk/authDb
                Creating: /opt/splunk/var/lib/splunk/hashDb
New certs have been generated in '/opt/splunk/etc/auth'.
        Checking critical directories...        Done
ERROR: pid 19199 terminated with signal 9
Validating databases (splunkd validatedb) failed with code '-1'.  If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue

[mikeconn · Accepted solution]

Just to update this, a security patch released yesterday seems to have corrected this. The kernel in 16.04 LTS is now on 4.13.0-32, released for USN-3548-2. Ubuntu 17.10 has the same fixes in USN-3548-1.

[louiseaxon]

This is resolved for me now - as you said @mikeconn, the latest update fixed it. I updated Ubuntu 16.04, so the kernel is now 4.13.0-32. The error has disappeared, and Splunk starts.

Thanks

[marthodder]

I've experienced this same issue after installing the latest patches in Ubuntu 17:10. Splunk now fails to start on any VM (both enterprise and universal forwarders) with the same error code. Not had time to investigate yet but i suspect its a doggy patch, possibly for the recent meltdown/spectre issues.

[mikeconn]

I'm getting exactly the same in Ubuntu 16.04 LTS. If I use the default boot, which on mine is 4.13.0-31, I get that failure. If I choose to boot an earlier kernel, 4.13.0-26 in this case, it works fine. Within Ubuntu, I'm actually running Splunk in CentOS containers, for demonstration purposes, and the affect of the Ubuntu kernel version goes through to them.