Hi everyone,
I am running Splunk 6.2.2 on a distributed setup with 3 search heads in a search head cluster and 4 non-clustered indexers. Splunk seems sluggish and I am trying to figure out why startup.handoff always seem to take a long time. For example:
This search has completed and has returned 595 results by scanning 806 events in 6.179 seconds.
Duration (seconds) Component Invocations Input count Output count
0.01 command.fields 14 595 595
0.02 command.remotetl 14 595 -
0.26 command.search 14 - 595
0.02 command.search.fieldalias 8 806 806
0.02 command.search.calcfields 8 806 806
0.01 command.search.index 14 - -
0.01 command.search.filter 8 - -
0.00 command.search.index.usec_1_8 1,431 - -
0.00 command.search.index.usec_8_64 25 - -
0.08 command.search.rawdata 8 - -
0.07 command.search.kv 8 - -
0.05 command.search.typer 8 595 595
0.03 command.search.lookups 8 806 806
0.01 command.search.tags 8 595 595
0.00 command.search.summary 14 - -
0.00 dispatch.check_disk_usage 1 - -
0.12 dispatch.createdSearchResultInfrastructure 1 - -
1.00 dispatch.evaluate 1 - -
1.00 dispatch.evaluate.search 1 - -
0.28 dispatch.fetch 15 - -
1.04 dispatch.finalizeRemoteTimeline 1 - -
0.00 dispatch.localSearch 1 - -
0.01 dispatch.parserThread 13 - -
0.02 dispatch.process_remote_timeline 2 109,035 -
0.17 dispatch.readEventsInResults 1 - -
0.04 dispatch.remote_timeline_fullevents 7 673,420 363
0.00 dispatch.stream.local 1 - -
0.28 dispatch.stream.remote 13 - 869,624
0.08 dispatch.stream.remote.SPLUNKIDX06 3 - 259,267
0.07 dispatch.stream.remote.SPLUNKIDX05 3 - 263,521
0.07 dispatch.stream.remote.SPLUNKIDX04 3 - 190,198
0.06 dispatch.stream.remote.SPLUNKIDX03 3 - 152,511
0.06 dispatch.timeline 15 - -
0.03 dispatch.writeStatus 8 - -
0.17 startup.configuration 6 - -
6.40 startup.handoff 6 - -
Any good tips where to look for the problem?
Thank you
... View more