topic Re: Splunk Admin searches being queued, but not running. in Splunk Search

Splunk Admin searches being queued, but not running.

djreschke — Fri, 06 Nov 2020 17:43:30 GMT

Good afternoon everyone,

I am the Splunk admin for our instance of Splunk, and yesterday later in the afternoon, I noticed that my searches were not running anymore, and that everything was being sent to the queue? In my troubleshooting efforts I signed into another searchhead, and ran a similar search with successful results.

Today, I thought what could be causing the issue was, i have a lot of saved searches that run with my username as the owner, so reassigned those knowledge objects, which did not fix the issue. I can see other users running searches, with no issue, so it appears to be my account.

Any advice of where to look next or has anyone experienced this issue before?

I am running 7.2.8 for on my searchheads and using my account not the local admin account.

Thank you.

Re: Splunk Admin searches being queued, but not running.

isoutamo — Fri, 06 Nov 2020 19:34:38 GMT

Check if your quotas has full. You could open jobs and remove some old finalized jobs and if your new jobs starts after this you know the reason. You could also check the situation from internal logs on node.

Re: Splunk Admin searches being queued, but not running.

bobd32 — Fri, 06 Nov 2020 20:41:23 GMT

Full restart!

Re: Splunk Admin searches being queued, but not running.

djreschke — Tue, 17 Nov 2020 12:09:59 GMT

Good morning everyone,

Sorry for the late update, but a full restart did not work and I checked the running jobs and I don't have any running, just 5 in queue, which they have been stopped. Checking internal logs this morning.

Re: Splunk Admin searches being queued, but not running.

isoutamo — Wed, 18 Nov 2020 16:30:54 GMT

And how many finalised/failed etc. jobs you have there waiting for purge?

Re: Splunk Admin searches being queued, but not running.

djreschke — Fri, 18 Dec 2020 12:27:59 GMT

Thanks Everyone for your feedback on this topic, I did have a few jobs that need purged, and I did perform a full restart, but it was a combination of the above comments/suggestions that help and have yet to experience the issue again.