topic Re: dispatch.finalizeRemoteTimeline taking a long time? in Splunk Search

dispatch.finalizeRemoteTimeline taking a long time?

dshpritz — Fri, 14 Feb 2014 18:58:33 GMT

When I run a search in Splunk 6.x, the results come back quickly, but it seems like a lot of time is spent on "Finalize Job". When I look at the search inspector, it shows that a lot of time is spend on "dispatch.finalizeRemoteTimeline". Is there anything I can do about that?

Thanks,

Dave

Re: dispatch.finalizeRemoteTimeline taking a long time?

martin_mueller — Fri, 14 Feb 2014 21:40:29 GMT

Are you using a Splunk 6 searchhead on a Splunk 5 indexer? If so, take a look at http://docs.splunk.com/Documentation/Splunk/6.0.1/DistSearch/Versioncompatibility#Remote_timeline_feature_can_reduce_performance

Re: dispatch.finalizeRemoteTimeline taking a long time?

dshpritz — Fri, 14 Feb 2014 21:42:38 GMT

Nope. Splunk 6 all the way around.

Re: dispatch.finalizeRemoteTimeline taking a long time?

a212830 — Fri, 18 Apr 2014 22:33:46 GMT

Same deal here - all Splunk 6. Search runs fine, but the finalizing takes a while.

Re: dispatch.finalizeRemoteTimeline taking a long time?

abhijitmishra — Mon, 28 Sep 2020 16:55:20 GMT

These were turned on by default in Splunk 6 for some reason. In the limits.conf on local folder, add these two configurations.

fetch_remote_search_log = false

remote_timeline_fetchall = false

Re: dispatch.finalizeRemoteTimeline taking a long time?

sibbsnb — Fri, 13 Mar 2015 07:42:24 GMT

What is the impact of turning this off?

Re: dispatch.finalizeRemoteTimeline taking a long time?

luhadia_aditya — Thu, 26 Mar 2015 06:23:19 GMT

What is the impact of turning this off?

Re: dispatch.finalizeRemoteTimeline taking a long time?

Lucas_K — Mon, 28 Sep 2020 19:32:30 GMT

I had the same thought as you so I looked into it. Turned both of them off and the change was AMAZING.

The "fetch_remote_search_log" option is the search.log files generated by each search peer. These are accessable via the job inspector at the very bottom. You will normally see them in the "Additional info" section. You can use these for troubleshooting but are generally not used very often at all. The more search peers you have the more of these search logs are created and sent back to the search head. Seems quite safe to turn off. Honestly this should probably be off by default and turned on for debugging purposes only.

The "remote_timeline_fetchall" im not 100% sure as the description is a little too vague. I'm guessing its the little bar chart in the flashtimeline/search bar. shrugs

Re: dispatch.finalizeRemoteTimeline taking a long time?

alacercogitatus — Thu, 18 Feb 2016 19:50:08 GMT

I know this is an old post, but this changes should take place on the SH, not needed on IDX. Just FYI for future me.

Re: dispatch.finalizeRemoteTimeline taking a long time?

bnorthway_splun — Tue, 14 Nov 2017 22:33:48 GMT

FYI this is no longer considered a "best practice" by Splunk Professional Services.

Re: dispatch.finalizeRemoteTimeline taking a long time?

aladda_splunk — Tue, 29 Sep 2020 17:34:21 GMT

@bnorthway - Are you saying setting these settings to false is no longer a PS best practice or the other way around?

fetch_remote_search_log = false
remote_timeline_fetchall = false

Re: dispatch.finalizeRemoteTimeline taking a long time?

bnorthway_splun — Fri, 05 Jan 2018 20:01:57 GMT

According to Sanford, it is not recommended to change this setting for versions after 6.1, 6.2, etc.

Re: dispatch.finalizeRemoteTimeline taking a long time?

anupkpal — Wed, 31 Jan 2018 18:25:37 GMT

Where do I find limits.conf file? Can some one send me the full path name of limits.conf file?

Thanks,

Anup Pal

Re: dispatch.finalizeRemoteTimeline taking a long time?

anupkpal — Wed, 31 Jan 2018 18:27:02 GMT

Can someone tell me the full path name of limits.conf file? Where is this .conf file located?

Thanks

Anup pal

Re: dispatch.finalizeRemoteTimeline taking a long time?

anupkpal — Wed, 31 Jan 2018 18:27:02 GMT

Can someone tell me the full path name of limits.conf file? Where is this .conf file located?

Thanks

Anup pal

Re: dispatch.finalizeRemoteTimeline taking a long time?

koshyk — Mon, 19 Feb 2018 10:09:57 GMT

it depends it you have cluster or standalone system. What we do is to put limits.conf within an app and push it to all Search heads.
So if you create an app & put in MY_sh_limits/local/limits.conf and these settings under

# $SPLUNK_HOME/etc/apps/MY_sh_limits/local/limits.conf 
[search]
fetch_remote_search_log = disabled
remote_timeline_fetchall = disabled

Re: dispatch.finalizeRemoteTimeline taking a long time?

koshyk — Tue, 29 Sep 2020 18:08:17 GMT

Just to make the answer more clear for those who want to test out. (It had quite significant change for us at 6.3.x version)

Create an app eg: MY_sh_limits and put these settings into "limits.conf"
Ultmiately it will be present in $SPLUNK_HOME/etc/apps/MY_sh_limits/default/limits.conf of your SH member.
Deploy it to SH cluster (via deployer) or manually if its a standalone. Please note, the values are "disabled" in newer versions.

[search]
fetch_remote_search_log = disabled
remote_timeline_fetchall = disabled
Check the difference in speed after implementing it. If its bad, just remove MY_sh_limits app. if cluster, just redeploy from deployer.

Re: dispatch.finalizeRemoteTimeline taking a long time?

pcsegal — Sat, 07 Apr 2018 15:41:20 GMT

What would be the recommended fix for this problem?

Re: dispatch.finalizeRemoteTimeline taking a long time?

anwarmian — Wed, 30 Sep 2020 05:06:04 GMT

I have not tried it but isn't "remote_timeline_fetchall = false" good for reporting searches instead of search that you run on search window?