Deployment Architecture

User profile web UI option "Restart background jobs when the Splunk software is restarted" what does it actually do?


This option exists under each users profile.

alt text

This seems to imply that if a user submits a long running (ie. 2 hour report) ah-hoc search via the gui, then performs a "send to background" (ie. adjusts job ttl to 7 days) that it will be restarted IF the search head restarts.

I was under the impression that the search head will wait for 6 mins or so before it starts killing off in process jobs.

In the server.conf there is the splunkd stop timeout.

splunkd_stop_timeout =
* The maximum time, in seconds, that splunkd waits for a graceful shutdown to
complete before splunkd forces a stop.
* Default: 360 (6 minutes)

What actually happens to this (and other) long running jobs when the admin issues a "splunk restart" and users have this setting enabled?

Is the search head storing this job somewhere between shutdown and restart?

What about paused jobs with long ttl's?

I've tried pausing a search and then a search head restart.

alt text

So it appears that in certain circumstances this actually works?

Why wouldn't this be, or is it, normal behavior on shutdown for all jobs regardless of the splunkd stop timeout?

Pause all running jobs.
Restart all previously running but paused jobs?

.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!