Splunk Search

integrating splunk with upstart (ubuntu/debian)

crazygir
Explorer

is there a recommended way to integrate splunk with upstart, or should this simply be ignored for splunk's built-in init/rc scripts?

thanks!

Tags (1)

gkanapathy
Splunk Employee
Splunk Employee

So, it turns out you can launch splunkd via "splunk start" in a non-exiting mode using the --nodaemon parameter:

./splunk start splunkd --nodaemon

This might make it a lot easier. It's unfortunately not actually in the official docs or the help, but it works.

The PID you care about really will just be the splunk start wrapper, so it kind of doesn't entirely do it to just look for that PID. Worse, if the splunkd process ends or fails, the wrapper doesn't exit, which is kind of fail. I guess with this, you can chose one problem or another.

Lowell
Super Champion

I messed around for several hours trying to make this work, but couldn't get what I was looking for. There are a couple of issues with this. One, upstart wants to directly launch the daemon processes, which doesn't play well with the "splunk start" command which really a wrapper that kicks off several background processes (splunkd and splunkweb).

So the options basically come down to (1) launching the all the various splunk daemons independently with separate upstart configurations, or (2) using upstart as a lame wrapper and simply sticking "splunks start" and "splunk stop" as pre-start and post-stop scripts at which point upstart wouldn't even know the currently status of the process. In older upstart versions you could manually specify a PID file, but that's gone away quite some time ago.

The problem with #1 is splunkd. Launching splunk web (which is really just cherrypy, which is really python) wouldn't be to bad, but launching splunkd manually would probably take some additional understanding of what the "splunk start" process does before actually launching the "splunkd" process, and chances are that could change a bit between versions of splunk, so even if you straced the whole thing and converted it into a nice little pre-start script in a upstart job, it could change next time you upgrade splunk, and some critical startup tasks could be missed. Which could be a big deal.

I stuck with the default (or mostly default) init.d script myself, but if anyone comes up with something better, I'd like to know about it too.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

So, it turns out you can launch splunkd via "splunk start" in a non-exiting mode using the --nodaemon parameter:

./splunk start splunkd --nodaemon

This might make it a lot easier. It's unfortunately not actually in the official docs or the help, but it works. Sort of.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

You could launch Splunk using upstart. Splunk will generate a default rc script and the corresponding links, but that is just a convenience and it doesn't depend on them at all. You can basically do whatever you like. If you want to generate the script, copy and use it for reference, then disable/delete it, that's fine. (Or ask someone who has a copy to send you one, whatever.)

All you have to do (and you will see this in the script) is simply run "$SPLUNK_HOME/bin/splunk start" as the correct user, and "$SPLUNK_HOME/bin/splunk stop" to stop, using whatever means you like.

0 Karma

crazygir
Explorer

Great!

anyone have a working copy to share with the world (there is surprisingly little about this topic available)?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Note that the command "$SPLUNK_HOME/bin/splunk start" starts up Splunk and then exits, so you'll have to take that into account in any scripting.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...