topic Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0? in Installation

Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

varad_joshi — Tue, 16 Jun 2020 18:55:46 GMT

I upgraded Splunk on my linux server to 6.5.0 and after that no searches are getting completed. Not even index=_internal

Search job inspector also doesn't load.

All the jobs are getting stuck at 'finalizing job'. Tried bouncing Splunk web services as I was occasionally getting error as "Connection to Splunk server lost."

Any solution please??

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

varad_joshi — Wed, 05 Oct 2016 08:48:07 GMT

I did a few things that includes:

Restarting of Splunkweb services as I was also getting an error 'Connection to Splunk server lost'
Clearing dispatch directory

Cleared everything fro dispatch directory.

And after a while everything started working. I am not sure if that was the issue.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

polymorphic — Wed, 05 Oct 2016 10:57:42 GMT

Didnt work for me 😕

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

varad_joshi — Wed, 05 Oct 2016 11:29:30 GMT

Hopefully someone will have a concrete answer.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

polymorphic — Wed, 05 Oct 2016 12:36:52 GMT

Now i actually tried to wipe the entire Splunk installation, and install from scratch.
(It is only a test installation)
Still having the same issue.

Could this maybe be an OS issue?
trying to update

No change after update.
Also tried the suggestion from @ddrillic. Still no change.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

ddrillic — Wed, 05 Oct 2016 13:17:14 GMT

A related discussion at dispatch.finalizeRemoteTimeline taking a long time?

It says there -

-- When I run a search in Splunk 6.x, the results come back quickly, but it seems like a lot of time is spent on "Finalize Job". When I look at the search inspector, it shows that a lot of time is spend on "dispatch.finalizeRemoteTimeline".

An answer by @abhijitmishra says -

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

varad_joshi — Wed, 05 Oct 2016 13:47:57 GMT

What OS you are running? I see no mention of Win 7 while downloading executable.
Also I already heard it giving issues on Win 7 32 bit machines.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

polymorphic — Wed, 05 Oct 2016 13:50:00 GMT

Using Debian Jessie 64bit.
Also tested on Windows 10 64bit. No problems here.

//Jesper S

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

polymorphic — Wed, 05 Oct 2016 13:58:58 GMT

Hmm seems to be the search app which gives the problems...
Searches run fine in other apps.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

varad_joshi — Wed, 05 Oct 2016 14:00:15 GMT

Oh yes.. I missed mentioning that.. Infact all dashboard also works fine

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

adepasquale — Wed, 05 Oct 2016 18:33:25 GMT

I too have this problem after upgrading from 6.3 to 6.5 on a windows server. All dashboards work fine, search just doesn't populate anything.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

varad_joshi — Wed, 05 Oct 2016 18:38:46 GMT

Please vote this question then. Might make an impact.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

adepasquale — Wed, 05 Oct 2016 18:58:02 GMT

Restarting did not work for me either. I was not getting disconnect errors though.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

adepasquale — Wed, 05 Oct 2016 20:59:09 GMT

The answer is embarrassingly simple... clear the cache and cookies for the browser you wish to use.

After updating two browsers (IE and edge) it worked. But firefox and chrome were on the latest builds. Simply clearing the cache and cookies worked on them.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

sgarvin55 — Wed, 05 Oct 2016 21:02:40 GMT

search is always "parsing...." after upgrading to 6.5 from 6.3.2 - SPL-129476

This is actually caused by stale files remaining in cache after the upgrade. If you haven't all ready done so, try clearing browser cache and retry the search. This has helped quite a few customers reporting this issue.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

apilgrim_splunk — Wed, 05 Oct 2016 21:03:48 GMT

The reason for the issue to disappear is because of browser cache clearing. If stale files remain in the cache after the upgrade we hit the issue.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

polymorphic — Thu, 06 Oct 2016 05:41:16 GMT

Thanks.
First time i ever had caching issues with Splunk.

Clearing browser cache solved the issue.

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

varad_joshi — Thu, 06 Oct 2016 05:45:18 GMT

Thanks for the answer @sgarvin55 and thanks @polymorphic for confirming the solution.

A colleague of mine who was accessing the same Splunk instance never cleared browser cache. Still it started working for her.

I had cleared it though on my browser.

Still a strange solution if we need to ask all users to clear their browser cache in order to use 6.5 😄

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

snix — Thu, 06 Oct 2016 15:52:09 GMT

"Slaps Head" I was in the same boat when going from 6.3 to 6.5 and just now tried clearing the browser cache for the local Spunk domain in Chrome by holding the control key and clicking on refresh wile on the page and it worked like a champ! Good catch @adepasquale

Re: Why are searches stuck at "finalizing job" after upgrade to 6.5.0?

varad_joshi — Thu, 06 Oct 2016 16:29:11 GMT

Ah more and more folks reporting this as solution.
Thanks again for confirming.

@Splunk should definitely look into it. It's a pain to ask my 200 users to clear cache of their browser.