topic Single/Multi Threading CPU in Deployment Architecture

Single/Multi Threading CPU

LCM — Fri, 25 Feb 2011 19:02:26 GMT

Is splunk itself single or multi threading capable in a CPU architecture perspective?

The docus I've read was mostly about x86 arch. but not SPARC!

Example:

If splunk is multithreaded and I'm installing it on a T-series server (Sun) I might be faster than on a M-series server (Sun), cos T-series has more threads per core. On the other hand, if it's singlethreaded, I'm better on the M-series (less threads, but more power per thread), but that one is much more expensive.

I know, who will install it these days on SPARC - sometimes politics wins!

I read through some documentation, and I might have a guess: Single Threaded!

Re: Single/Multi Threading CPU

David — Sat, 26 Feb 2011 02:00:40 GMT

Take a look at this Splunk Answers:

http://answers.splunk.com/questions/1829/splunk-searches-to-be-multithreaded-in-a-single-box

The short answer is that Splunk as a system is multi-threaded, but there are some Splunk processes that are not. Notably: indexing is multi-threaded, searches aren't. As gkanapathy mentions below, each search will use a single process and thread, so if you have 12 simultaneous searches, and a high indexing load, Splunk itself could use 16 threads.

Perhaps an engineer can provide guidance if there is anything particular to SPARC systems, but I believe that link is representative.

Re: Single/Multi Threading CPU

LCM — Sat, 26 Feb 2011 23:25:31 GMT

In the thread you posted, silvermails answer explains it best (for me). While searching, one core is beeing clamed and the rest ist idling (single-thread) - Thanks David for the hint!

Re: Single/Multi Threading CPU

gkanapathy — Sun, 27 Feb 2011 06:56:23 GMT

That is incorrect. The indexing process is multi-threaded, up to four or more threads, though in practice you'll usually see two.

Re: Single/Multi Threading CPU

gkanapathy — Sun, 27 Feb 2011 06:56:48 GMT

Furthermore, each concurrent search uses a full process and thread.

Re: Single/Multi Threading CPU

gkanapathy — Sun, 27 Feb 2011 06:58:22 GMT

All that said, please do not analogize Splunk with a web server for the purposes of sizing and multithreading. It is much more analogous to a database server, in that CPU and threads are only a small part of the performance factors, and rarely the most important ones.

Re: Single/Multi Threading CPU

David — Tue, 01 Mar 2011 02:25:45 GMT

Thanks for the corrections. I fixed the errors in my original answer, should anyone read it in the future and skip over comments.