Installing Enterprise Security on a Search Head Cluster / Index Cluster

[harrymclaren]

Hi Team,

I have a brand new Splunk implementation. Both SH Cluster and IX Cluster are setup and supported by a Deployer and Cluster Master respectively. The SH Cluster is forwarding all data to the IX Cluster along with UFs etc.

I'm now at the stage where ES need to be installed, I have found the documentation to install a ES into a new implementation very minimal and confusing with regard to a clustered environment.

Do I just need to add the ES app to the Deployer (/apps/enterprise_security) which will install on the SHs within the cluster, then install the relevant ES app onto the Cluster Master (/master-apps/enterprise_security) to install it on all IXs?

Thanks for your help.

[rsennett_splunk · Accepted solution]

You'll want to read this document very carefully: http://docs.splunk.com/Documentation/ES/3.3.0/Install/AdvancedImp It details how to implement ES on a Search Head Cluster.

If you haven't installed ES before (usually Professional Services does the install, since there are a lot of moving parts to hook up), you may wan to follow maciep's advice and initially set ES up as a Standalone, and the move the SA's DA's and TA's to the proper parts of your deployment (they all go in different spots) especially since the Deployer and Cluster Master will need to receive the different chunks. This is normally a two to three week Professional Services Engagement, simply because each element must be carefully configured and tested for all the moving parts to work out of the gate. The extended time is also used for custom creation of Add-Ons using the CIM. In addition, the experience of having installed ES multiple times (Sometimes hundreds) helps the PS consultants quickly tackle any sort of wrench thrown into the works.

Configuring ES is a repetitive process. For each data type you will go through this checklist as documented:
1. Normalize the data indexed in Splunk Enterprise to the CIM.
2. Define, then configure Splunk App for Enterprise Security user roles.
3. Collect, process, and import the asset and identities information.
4. Collect, process, and import threat lists, or other sources of security information.
5. Review and enable correlation searches for the security domains that contain data.
6. Customize the Enterprise Security navigation settings.

If you start off in a "test" environment, you'll get some practice with going through the process and may find that mid way through, you're ready to start incorporating it into your production environment.

I've been at Splunk for three years(not in PS) ... and truthfully, I wouldn't want to do it myself... and I will stand on my head and sing arias for my customers, to support the concept of having PS do the install, because I know - that if it were MY ES deployment, I'd have wanted it to be implemented by an expert who has "seen it all".

Just my two cents...

[maciep]

At least with the upgrade, the idea is to stage the it on a standalone instance and then copy back to deployer. Maybe the same thing with a new install?

Bring up a new splunk server, with nothing installed. Install/Configure ES. Once ready, stop splunk and copy the ES-related apps over to the deployer. Then push them out to the cluster.

Probably a little more complicated than that but I'm sure you get the idea. We migrated to our cluster from a standalone implementation, so we never had to do a brand new install new in our cluster.

[harrymclaren]

So I installed ES on the Deployer (as it has no other apps locally installed).
However, the are 10's of apps now in the local app directory. Would I only copy the relevant ES apps I need to the deployment directory? Is the a list of what each one is for?

Also is the a list of which apps I should be copying over to the Cluster Master for deployment the indexers?

Thanks!

[maciep]

There are a lot of SA's, DA's and TA's associated with ES. I believe they should all be on the deployer. Essentially, anything that got put down during the install/upgrade should be placed on the deployer. Obviously, don't copy over the default Splunk apps (like search, learned, etc)

I'm surprised there's not more documentation on the indexer cluster. But yes, again, for the TA's, DA's and SA's there could be indexes, index-time config, etc that would need to be pushed down from your cluster master. So you'd want to copy those apps over as well.

In general, this is a pretty complicated app and we had Professional Services available to help us through it. So I'm familiar, but not an expert.

List of indexes: http://docs.splunk.com/Documentation/ES/latest/Install/Indexes
Search Head Cluster: http://docs.splunk.com/Documentation/ES/latest/Install/AdvancedImp

[wahmad_splunk]

also, check out http://docs.splunk.com/Documentation/ES/3.3.0/Install/DeploymentPlanning