I have a saved search that runs in roughly 3-5 mins but then hangs for 5+ hours on finalizing. Running Splunk 7.0.1 and below is the job inspection. Any ideas on making the finalize step quicker?
Duration (seconds) Component Invocations Input count Output count
156.07 .execute_input.flush_prestats 102 34,419,541 34,419,541
445.15 command.tstats 355 34,830,486 34,830,486
282.35 command.tstats.query_tsidx 173 - -
162.77 command.tstats.execute_input 181 34,830,486 -
0.02 dispatch.check_disk_usage 19 - -
0.01 dispatch.createdSearchResultInfrastructure 1 - -
0.02 dispatch.evaluate 7 - -
0.02 dispatch.evaluate.rename 21 - -
0.01 dispatch.evaluate.eval 14 - -
0.01 dispatch.evaluate.collect 7 - -
0.01 dispatch.evaluate.fields 7 - -
0.01 dispatch.evaluate.lookup 7 - -
0.01 dispatch.evaluate.spath 7 - -
0.01 dispatch.evaluate.stats 7 - -
0.01 dispatch.evaluate.tstats 7 - -
0.01 dispatch.evaluate.where 7 - -
0.00 dispatch.evaluate.noop 7 - -
15.49 dispatch.fetch 181 - -
0.00 dispatch.localSearch 1 - -
0.03 dispatch.optimize.FinalEval 7 - -
1.43 dispatch.optimize.matchReportAcceleration 7 - -
0.26 dispatch.optimize.optimization 7 - -
0.01 dispatch.optimize.reparse 7 - -
0.02 dispatch.optimize.toJson 7 - -
0.01 dispatch.optimize.toSpl 7 - -
75.54 dispatch.parserThread 179 - -
0.00 dispatch.stream.local 1 - -
282.37 dispatch.stream.remote 173 - 12,503,665,064
86.68 dispatch.stream.remote.pspl002.amfam.com 38 - 3,239,089,995
85.26 dispatch.stream.remote.pspl001.amfam.com 35 - 3,013,629,934
56.61 dispatch.stream.remote.pspl011.amfam.com 41 - 3,144,605,731
53.16 dispatch.stream.remote.pspl012.amfam.com 41 - 3,091,705,296
0.35 dispatch.stream.remote.pspl014.amfam.com 10 - 7,423,872
0.32 dispatch.stream.remote.pspl013.amfam.com 8 - 7,210,236
0.45 dispatch.writeStatus 117 - -
1.09 startup.configuration 19 - -
9.01 startup.handoff 19 - -
