Deployment Architecture

what is needed to speed up searhing? indexer cluster or search head cluster?

highsplunker
Contributor

he guys! i have 6 linux servers. i'm trying to understand what is needed to speedup searching: indexer cluster or search head cluster?

0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @highsplunker,
so generally said probably both, in any case the presence or absence of a cluster is relevant only if you have high affidability needs otherwise it is not needed.
In order to make a search faster, in general you need at first fast disks (Splunk requests at least 800 IOPS ore more).

Then the presence of many servers (indexers or Search Heads) is relevant based on the load: how many users will use the system? how many contemporary searches will be launched and/or scheduled?
Keep in mind that every search (and every sub-search) takes a CPU, so many servers with many CPUs are useful if there are a lot of searches at the same time.
When you size your system keep in mind the minimum dimensional parameters (which you can find at https://docs.splunk.com/Documentation/Splunk/8.0.3/Capacity/Referencehardware ).
Finally, you need to see how searches are done (for example, an extensive use of transactions and joins makes searches very slow).

In conclusion, first you need to do an analysis of the requirements of your system:

  • high reliability yes / no,
  • volume of data indexed daily and at peak times,
  • system users,
  • scheduled searches, contemporary searches.

I hope I managed to give you an initial overview of the problem that requires careful analysis and design made by an experienced Splunk Architect.

Ciao.
Giuseppe

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @highsplunker,
so generally said probably both, in any case the presence or absence of a cluster is relevant only if you have high affidability needs otherwise it is not needed.
In order to make a search faster, in general you need at first fast disks (Splunk requests at least 800 IOPS ore more).

Then the presence of many servers (indexers or Search Heads) is relevant based on the load: how many users will use the system? how many contemporary searches will be launched and/or scheduled?
Keep in mind that every search (and every sub-search) takes a CPU, so many servers with many CPUs are useful if there are a lot of searches at the same time.
When you size your system keep in mind the minimum dimensional parameters (which you can find at https://docs.splunk.com/Documentation/Splunk/8.0.3/Capacity/Referencehardware ).
Finally, you need to see how searches are done (for example, an extensive use of transactions and joins makes searches very slow).

In conclusion, first you need to do an analysis of the requirements of your system:

  • high reliability yes / no,
  • volume of data indexed daily and at peak times,
  • system users,
  • scheduled searches, contemporary searches.

I hope I managed to give you an initial overview of the problem that requires careful analysis and design made by an experienced Splunk Architect.

Ciao.
Giuseppe

highsplunker
Contributor

hi Giuseppe! @gcusello !
thanks for quick an detailed response!
my question is, probably, with this:
Keep in mind that every search (and every sub-search) takes a **CPU* * - CPU where?

if it's a CPU on indexer server, then i probably need indxer cluster
if it's a CPU on search head server, then search head

am i correct?

best regards
Rashid

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @highsplunker,
As I said cluster is needed if you want HA, otherwise you don't need it and you can use the same number of non clustered Indexers.
A search takes a CPU both on Indexers and Search Heads, but the problem is the Indexers.

The important thing is to understand, based on your users, ingestion and search needs, how many CPUs you need.
As you can read in Hardware references, you need at least 12 CPUs on each Indexer and 16 CPUs on each Search Head, but how many Indexers and how many Search Heads you need depends on how many concurrent searches and ingestions you have.

Did you analyzed, the daily and peak volume of data to ingest?
Did you analyzed the number of concurrent searches you are waitng for?
With these informations you can choose the correct resources for your system.

Remember to have quick disks for the Indexers: no RAID5, only RAID 1+0, at least SAS 15k to have at least 800 IOPS.
Remember that a SAS 15k disk has around 100-150 IOPS, so it's better to have more disks.

Ciao.
Giuseppe

0 Karma

highsplunker
Contributor

ok, thank you Giuseppe!
I'm not sure I understand my current needs fully, so I'm going to analyze carefully again.

thank you!

best wishes,
rashid

0 Karma
Get Updates on the Splunk Community!

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...

Splunk and Fraud

Watch Now!Watch an insightful webinar where we delve into the innovative approaches to solving fraud using the ...