Deployment Architecture

User profile web UI option "Restart background jobs when the Splunk software is restarted" what does it actually do?

Lucas_K
Motivator

This option exists under each users profile.

alt text

This seems to imply that if a user submits a long running (ie. 2 hour report) ah-hoc search via the gui, then performs a "send to background" (ie. adjusts job ttl to 7 days) that it will be restarted IF the search head restarts.

I was under the impression that the search head will wait for 6 mins or so before it starts killing off in process jobs.

In the server.conf there is the splunkd stop timeout.

splunkd_stop_timeout =
* The maximum time, in seconds, that splunkd waits for a graceful shutdown to
complete before splunkd forces a stop.
* Default: 360 (6 minutes)

What actually happens to this (and other) long running jobs when the admin issues a "splunk restart" and users have this setting enabled?

Is the search head storing this job somewhere between shutdown and restart?

What about paused jobs with long ttl's?

I've tried pausing a search and then a search head restart.

alt text

So it appears that in certain circumstances this actually works?

Why wouldn't this be, or is it, normal behavior on shutdown for all jobs regardless of the splunkd stop timeout?

Pause all running jobs.
restart.
Restart all previously running but paused jobs?

tschikste
Engager

Does anyone have an answer to that?

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...