topic Re: Best Practices for Multi-User Search Performance in Getting Data In

Best Practices for Multi-User Search Performance

balbano — Sat, 22 May 2010 00:00:46 GMT

Hey guys,

I currently have a 3-server architecture (2 central indexers with 1 search head). We are looking to have Splunk used by multiple sys admins and developers in the company. I foresee that we could have approx 10+ user sessions going on at the same time. I am currently testing out having my test sample set of Splunk LF Deployed Clients send data which is load-balanced across the 2 indexers. Also am aware of time-restricted searches for some user accounts as well as summary indexing.

From a hardware perspective, what would be the preferred specs to handle up to 10+ users simultaneously?

Also, does there need to be any specific RAID Configurations?

The reason I am asking all of this is because it seems like when more than 3 people run simultaneous searches seems like Splunk starts to choke.

Any help you can provide on this would be great.

Thanks.

Brian

Re: Best Practices for Multi-User Search Performance

hulahoop — Sat, 22 May 2010 03:37:48 GMT

Hi Brian,

Have you already discovered the Capacity Planning topic on the Community Wiki?

The general rule for sizing search head capacity is 1 core per active user, but the capacity of your indexing servers also plays an important role. In any case, the link above reviews the interplay between search head and indexers and additional considerations for sizing each. It also provides specs for Splunk recommended hardware.

Re: Best Practices for Multi-User Search Performance

gkanapathy — Sat, 22 May 2010 08:07:24 GMT

It would be helpful if you also provided specs on the current machines, particularly CPU and core count, and the type of disk/RAID the indexes are on. Also if you could quantify your approximate data volume and amount of summarization that would let us provide more specific answers.

Re: Best Practices for Multi-User Search Performance

gkanapathy — Sat, 22 May 2010 08:08:14 GMT

But given what you've said, either your indexer disks are too slow, or your machines don't have enough CPU cores.

Re: Best Practices for Multi-User Search Performance

balbano — Tue, 25 May 2010 01:58:17 GMT

I think its a combination of mismatch of hardware between the 2 indexers, one having fairly newer hardware and the other being older, and the RAID Configurations, I believe our indexers have RAID 5 configuration instead of recommended recommendation of RAID 10...

Guess that answers my question... thanks...

Re: Best Practices for Multi-User Search Performance

balbano — Tue, 25 May 2010 04:17:15 GMT

Here are the specs they pretty much show the hardware inconsistency:

indexer 1:

4 CPU Cores: Intel(R) Xeon(R) CPU X5270 @ 3.50GHz
Memory: 4GB
RAID: RAID 5

indexer2:

16CPU Cores: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
Memory: 16GB
RAID: RAID 5

search head:

4 CPU Cores: Dual-Core AMD Opteron(tm) Processor 2220 @ 2.8GHz
Memory: 32GB

Re: Best Practices for Multi-User Search Performance

hulahoop — Sat, 12 Jun 2010 00:56:51 GMT

One of our Solution Architects told me once--there's no good reason to use RAID 5 or 6. Splunk suffers pretty bad performance under these RAID configurations. Glad you got to the bottom of this and posted your specs for reference.