disable webserver ignored on AIX?

[jpdubose]

We are using SplunkLightForwarder on an AIX box. We don't want port 8000 listening and have no real need for the web interface on the box so we have run "splunk disable webserver" to prevent splunkweb from starting. As such, there is a web.conf file in etc/system/local and it has the proper "startwebserver = 0" line.

This is where it gets kind of strange. On a boot or reboot of the server, splunkweb starts and is listening on port 8000. You can then go to $SPLUNK_HOME/bin and issue a "./splunk restart" command and splunkd and splunkweb stop and then only splunkd is started back up.

Is there something with inittab or somewhere else that splunk starts on machine boot that is still starting splunkweb or is this some kind of bug?

This was the behavior with splunk 3.4 and now with 4.1.2.

Thanks

[mitch · Accepted solution]

Yeah, looks like a bug. On AIX, "splunk enable boot-start" adds splunkd and splunkweb to SRC. Unfortunately the way that it does it probably won't respect the "disable webserver" setting because once its in SRC, bin/splunk isn't the one doing the launching and it doesn't get a chance to apply that setting. I'll have to think about how to handle this one.

It should be easy to manually fix on the box if you just do a:

  # rmssys -s splunkweb

to remove splunkweb from SRC

[jpdubose]

The rmssys command did the trick.

So do I get a shirt for finding a bug? 🙂

[Mick]

It sounds like there's something in your server's startup prcoess that calls both the splunkd and splunkweb services directly. Perhaps something was configured originally to do this before the SplunkLightForwarder app was enabled?

When you just run the ./splunk start or ./splunk restart commands, it sounds like Splunk itself is doing the right thing, and not starting the splunkweb process, so something else must be calling that service directly. What have you got configured to start Splunk automatically when the server boots?

[dwaddle]

This sounds like a nonstandard part of the Splunk on AIX install. Splunk by default does not register itself with SRC, and would not be startable using startsrc. If you can, please post the output of the command (run as root):

odmget -q grpname=splunk SRCsubsys

[jpdubose]

It looks like Splunk is started in the inittab through the following line:

splunk:2:once:/usr/bin/startsrc -g splunk > /dev/console 2>$1