Monitoring Splunk

Best distributed search Architecture for Big amount of data per Day

MarMoh
Path Finder

Hi All,

Currently we are running a stand alone Enterprise version of Splunk(500M/day) on a Windows server. Since Our Company is grwoing we are expectiong to have gigs of data daily. We decided to deploy distributed search but I'm so new to this and wanted to see if any one has any practical architecture to go for and I also wanted to know if I can use this stand alone instance as a Head search or peer?

Regards,
M

Tags (1)
0 Karma
1 Solution

Jon_Webster
Splunk Employee
Splunk Employee

The Splunk Distributed Deployment Manual has all the details on this, but if you're using a server-class system you may not need additional servers until you exceed 100-200GB/day.

The simplest sizing guide in the Manual is this page, and I recommend reading the whole "Estimate hardware requirements" subject starting here.

Yes, you can use your existing Splunk server as a Search Head or Search Peer if you expand to a distributed environment.

The first step that most people take is adding a second indexer to their environment. You would keep your existing Splunk server as it is, and add another server setup as an indexer only, configure the original server to search both it's local data and the new server's data, and configure any Splunk Forwarders to auto load balance and send data to both indexers. This would basically double the amount of data you can index daily, and increase your search performance.

View solution in original post

MarMoh
Path Finder

Thanks Jon but I have tons of questions now 🙂
So I decided to use the existing one as my indexer and ad a VM search head cuz I talked to my manager and he is not happy with 2 physical boxes so Ill set up a VM as a dedicated search head.
Now my questions:
1- is it even a good idea to use VM as a dedicated search head?
2-in the chrat number of search users are too low. Data volume wise 1 head would be enough for us but number of search users is only 4! right now too many people are using the current Splunk at the same time!
3-how much work does it take to add another search head in future! How is it going to impact the end users when we are doing it?
4-How end users can access the splunk if we have multiple search heads or indexers.right now we just acces https://splunk.

We are so concerned about scailability and the possible impact. We'd rather configure 2 search heads now rathater than next year if it impacts our end users!

0 Karma

lmyrefelt
Builder

You proberbly would want to configure another indexer as you will spread Your load better, 4 users for a search head With the right specs should be enough for 4 users ( well depending of course of number and how Heavy the searches are) .

2 indexer means "less data" for them to ingest but more disks and cpus to use to search the data.

check the cpu / memory and disk usage (i/o wait time) on both the indexer and search head to check where you should put Your effort.

I am sure you can handle one server for 100GBs indexing for a day as long as you dont do any searches 😉

0 Karma

MarMoh
Path Finder

Thank you so much Jon.

0 Karma

Jon_Webster
Splunk Employee
Splunk Employee

The Splunk Distributed Deployment Manual has all the details on this, but if you're using a server-class system you may not need additional servers until you exceed 100-200GB/day.

The simplest sizing guide in the Manual is this page, and I recommend reading the whole "Estimate hardware requirements" subject starting here.

Yes, you can use your existing Splunk server as a Search Head or Search Peer if you expand to a distributed environment.

The first step that most people take is adding a second indexer to their environment. You would keep your existing Splunk server as it is, and add another server setup as an indexer only, configure the original server to search both it's local data and the new server's data, and configure any Splunk Forwarders to auto load balance and send data to both indexers. This would basically double the amount of data you can index daily, and increase your search performance.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...