I see. In this case this is only the search head that has 24 cores, and I am assuming the indexer is doing most of the work anyway, in reading through your Splunk presentation. So really this box is waiting for Network I/O, because it forks a few splunkd instances, which then makes REST calls to the indexer.
Due to the horizontal scaling architecture of Splunk, it is subtle to figure out how you can exactly speed things up. In our case what we really want to speed up is the number of events per second piped into a timechart. Will create another question about this.
... View more