Hi Splunk members,
How Can I get some metrics to indicate things like search concurrency, search queue depth, cancelled/timed out searches, etc by search head and by indexer?
have you read about the monitoring console http://docs.splunk.com/Documentation/Splunk/latest/DMC/DMCoverview yet?
You can try using metrics.log for various parameters.
Eg: index=_internal source=*metrics.log group=search_concurrency "system total" OR you can also use audit logs.
Let me know if this helps!!