Deployment Architecture

Need suggestions to deploy HA Splunk Architecture?

vikas_gopal
Builder

Hi Experts,

I need some suggestions on my Splunk Deployment,like what architecture fits for me .

Total Data volume :- 65GB /day
Concurrent users :- 10

Usage :- Scheduled reporting and searching
Data Sources :- Linux , Windows (total 80 servers)
Log Type :- Security , Event logs from Windows and Linux servers.

Following was the purposed wonderful architecture which seems very high end architecture , please suggest which part of the following I can remove or reduce to 4 servers else it would be very costly architecture.

Search Head (3 servers, 8 cpu, 15GB RAM, 500GB Disk, Genral SSD each)
Peers (2 Servers , 8cpu , 15 GB RAM, 8000GB, Genral SSD each )
Master & Deployment Server (1Server ,2CPU,4GB RAM,100GB Disk , Genral SSD)
Forwarders (2 Servers, 2CPU, 8GB RAM, 250 GB Disk, Genral SSD each)
SHC Deployer (1 server , 1CPU, 1GB RAM, 100GB Disk, Genral SSD)

Thanks
VG

0 Karma
1 Solution

jkat54
SplunkTrust
SplunkTrust

Vikas,

You can remove the forwarders and SHC Deployer and that's it if you want full HA. The CM/LM/DS would have to be the SHC Deployer. This is not a best practice architecture though.

I don't see the ELB mentioned here though so don't forget you'll need that for load balancing across the search head cluster.

You should also buy reserved instances for this which should be cheaper than on-demand instances.

View solution in original post

vikas_gopal
Builder

Hi Guys,

Really Appreciated both of your efforts , @jkat54 Actually I forgot to mention load balancer here so you are right we have one LB too. Now, requirement is to have full HA architecture , I guess to have 2 forwarders here to balance the load from 80+ destination servers . So do you recommend to remove completely these 2 forwarders ? So as per my understanding also recommended by @inventsekar can I opt following

Total 6 serves
Search head =2
Indexer=2
master and deployement servers =1
Load balancer =1

Thanks
VG

0 Karma

inventsekar
Ultra Champion

Hi Vikas, I would like to clarify few more details offline with you.. can you please send me an email please ..i checked ur profile for ur id, but I didn't find it.. my email id is in profile.

0 Karma

jkat54
SplunkTrust
SplunkTrust

Please refrain from asking members of the community to contact you via email. This is both unhelpful to the user as well as the community.

0 Karma

jkat54
SplunkTrust
SplunkTrust

The forwarders are not required to have HA inputs. Instead you will need to install the Universal Forwarder on all the servers sending data in and there you will configure an outputs.conf with the 'autoLB=true' setting and both indexers mentioned.

As for search heads, minimum search heads for SHC is 3.

It will operate with just 2, but it can’t elect a captain with RAFT when there are two or less search heads.

A captain is required for the knowledge object replication (aka search/alerting HA).

http://docs.splunk.com/Documentation/Splunk/6.5.0/DistSearch/SHCsystemrequirements#Summary_of_key_re...
(see the 4th point)

0 Karma

inventsekar
Ultra Champion

3 search heads are not needed, i hope.
maybe, 1 ... at the max 2 is enough, i hope.

for reference, i have saved this pic from Splunk docs, i hope.

alt text
alt text

jkat54
SplunkTrust
SplunkTrust

You cannot have full HA with 1 search head.

0 Karma

inventsekar
Ultra Champion

ya, actually, for 65GB per day data volume, HA / SHC is not needed i hope.
we are having around 800GB environment, without clustering and its working pretty good only.

alt text

0 Karma

jkat54
SplunkTrust
SplunkTrust

I believe Vikas requires HA. He titled the question "Need suggestions to deploy HA Splunk Architecture?"

0 Karma

jkat54
SplunkTrust
SplunkTrust

Vikas,

You can remove the forwarders and SHC Deployer and that's it if you want full HA. The CM/LM/DS would have to be the SHC Deployer. This is not a best practice architecture though.

I don't see the ELB mentioned here though so don't forget you'll need that for load balancing across the search head cluster.

You should also buy reserved instances for this which should be cheaper than on-demand instances.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...