Best distributed search Architecture for Big amount of data per Day

[MarMoh]

Hi All,

Currently we are running a stand alone Enterprise version of Splunk(500M/day) on a Windows server. Since Our Company is grwoing we are expectiong to have gigs of data daily. We decided to deploy distributed search but I'm so new to this and wanted to see if any one has any practical architecture to go for and I also wanted to know if I can use this stand alone instance as a Head search or peer?

Regards,
M

[MarMoh]

Thanks Jon but I have tons of questions now 🙂
So I decided to use the existing one as my indexer and ad a VM search head cuz I talked to my manager and he is not happy with 2 physical boxes so Ill set up a VM as a dedicated search head.
Now my questions:
1- is it even a good idea to use VM as a dedicated search head?
2-in the chrat number of search users are too low. Data volume wise 1 head would be enough for us but number of search users is only 4! right now too many people are using the current Splunk at the same time!
3-how much work does it take to add another search head in future! How is it going to impact the end users when we are doing it?
4-How end users can access the splunk if we have multiple search heads or indexers.right now we just acces https://splunk.

We are so concerned about scailability and the possible impact. We'd rather configure 2 search heads now rathater than next year if it impacts our end users!

[lmyrefelt]

You proberbly would want to configure another indexer as you will spread Your load better, 4 users for a search head With the right specs should be enough for 4 users ( well depending of course of number and how Heavy the searches are) .

2 indexer means "less data" for them to ingest but more disks and cpus to use to search the data.

check the cpu / memory and disk usage (i/o wait time) on both the indexer and search head to check where you should put Your effort.

I am sure you can handle one server for 100GBs indexing for a day as long as you dont do any searches 😉

[MarMoh]

Thank you so much Jon.

[Jon_Webster · Accepted solution]

The Splunk Distributed Deployment Manual has all the details on this, but if you're using a server-class system you may not need additional servers until you exceed 100-200GB/day.

The simplest sizing guide in the Manual is this page, and I recommend reading the whole "Estimate hardware requirements" subject starting here.

Yes, you can use your existing Splunk server as a Search Head or Search Peer if you expand to a distributed environment.

The first step that most people take is adding a second indexer to their environment. You would keep your existing Splunk server as it is, and add another server setup as an indexer only, configure the original server to search both it's local data and the new server's data, and configure any Splunk Forwarders to auto load balance and send data to both indexers. This would basically double the amount of data you can index daily, and increase your search performance.