dispatch.finalizeRemoteTimeline taking a long time?

[dshpritz]

When I run a search in Splunk 6.x, the results come back quickly, but it seems like a lot of time is spent on "Finalize Job". When I look at the search inspector, it shows that a lot of time is spend on "dispatch.finalizeRemoteTimeline". Is there anything I can do about that?

Thanks,

Dave

[koshyk]

Just to make the answer more clear for those who want to test out. (It had quite significant change for us at 6.3.x version)

• Create an app eg: MY_sh_limits and put these settings into "limits.conf"
  Ultmiately it will be present in $SPLUNK_HOME/etc/apps/MY_sh_limits/default/limits.conf of your SH member.

• Deploy it to SH cluster (via deployer) or manually if its a standalone. Please note, the values are "disabled" in newer versions.

  [search]
  fetch_remote_search_log = disabled
  remote_timeline_fetchall = disabled

• Check the difference in speed after implementing it. If its bad, just remove MY_sh_limits app. if cluster, just redeploy from deployer.

[sibbsnb]

What is the impact of turning this off?

[anupkpal]

Can someone tell me the full path name of limits.conf file? Where is this .conf file located?

Thanks

Anup pal

[anupkpal]

Can someone tell me the full path name of limits.conf file? Where is this .conf file located?

Thanks

Anup pal

[koshyk]

it depends it you have cluster or standalone system. What we do is to put limits.conf within an app and push it to all Search heads.
So if you create an app & put in MY_sh_limits/local/limits.conf and these settings under

# $SPLUNK_HOME/etc/apps/MY_sh_limits/local/limits.conf 
[search]
fetch_remote_search_log = disabled
remote_timeline_fetchall = disabled

[bnorthway_splun]

FYI this is no longer considered a "best practice" by Splunk Professional Services.

[aladda_splunk]

@bnorthway - Are you saying setting these settings to false is no longer a PS best practice or the other way around?

fetch_remote_search_log = false
remote_timeline_fetchall = false

[bnorthway_splun]

According to Sanford, it is not recommended to change this setting for versions after 6.1, 6.2, etc.

[pcsegal]

What would be the recommended fix for this problem?

[Lucas_K]

I had the same thought as you so I looked into it. Turned both of them off and the change was AMAZING.

The "fetch_remote_search_log" option is the search.log files generated by each search peer. These are accessable via the job inspector at the very bottom. You will normally see them in the "Additional info" section. You can use these for troubleshooting but are generally not used very often at all. The more search peers you have the more of these search logs are created and sent back to the search head. Seems quite safe to turn off. Honestly this should probably be off by default and turned on for debugging purposes only.

The "remote_timeline_fetchall" im not 100% sure as the description is a little too vague. I'm guessing its the little bar chart in the flashtimeline/search bar. shrugs

[abhijitmishra · Accepted solution]

These were turned on by default in Splunk 6 for some reason. In the limits.conf on local folder, add these two configurations.

fetch_remote_search_log = false

remote_timeline_fetchall = false

[alacercogitatus]

I know this is an old post, but this changes should take place on the SH, not needed on IDX. Just FYI for future me.

[anwarmian]

I have not tried it but isn't "remote_timeline_fetchall = false" good for reporting searches instead of search that you run on search window?

[luhadia_aditya]

What is the impact of turning this off?

[anupkpal]

Where do I find limits.conf file? Can some one send me the full path name of limits.conf file?

Thanks,

Anup Pal

[a212830]

Same deal here - all Splunk 6. Search runs fine, but the finalizing takes a while.

[dshpritz]

Nope. Splunk 6 all the way around.

[martin_mueller]

Are you using a Splunk 6 searchhead on a Splunk 5 indexer? If so, take a look at http://docs.splunk.com/Documentation/Splunk/6.0.1/DistSearch/Versioncompatibility#Remote_timeline_fe...