Deployment Architecture

Consolidating different search heads/SHC into a single SHC

dm1
Contributor

I have got a complicated task of consolidating two standalone search heads and a single search head cluster (4 nodes) all into a single search head cluster of 3 nodes.

 

Can someone please advise what would be the most efficient and correct method to accomplish this ?

Tags (1)
0 Karma

PickleRick
Ultra Champion

What do you mean by "consolidating" since you have 6 hosts and want to reduce this number to 3?

0 Karma

dm1
Contributor

Meaning migrating the search artifacts of those 3 search heads(1 being the SHC) into the new single SHC.

Basically, right now there are people using SHC for Reporting, 1 SH for purely dashboards, and 1 for adhoc searching, all of which are on-prem. I want them to just use one SHC for all those scenarios. This SHC will be newly built in self managed Splunk cloud environment.

0 Karma

PickleRick
Ultra Champion

You don't have "3 search heads(1 being the SHC)". You have 6 SHs and have to check what deployment mode you use on the SHC 🙂

It's an annoying process because you have to migrate all roles, users (you might have this already "done" if you use external auth), apps, user data (which can be tricky if you want to merge settings from different SHs into one SHC). There's no automatic way to do that, unfortunately. It's a painful and tedious manual process.

 

0 Karma

dm1
Contributor

Yes, my bad. From the point of view of unique search artifacts, I meant to say there are 3 SHs (considering nodes in SHC will contain same artifacts.)


@PickleRick wrote:

 user data (which can be tricky if you want to merge settings from different SHs into one SHC)


Thats the key part I am concerned about and wondering how I would go about properly migrating them. 

Can you please share how the manual process of this would like ?

I am aware its mainly a matter of copying over the etc/apps and etc/users directories. Luckily, there are no conflicting apps but there are definitely same users across all these SHs. Mainly, I 'd appreciate if you can tell me whats the best way do that ? 

 

0 Karma

PickleRick
Ultra Champion

The general idea is that you indeed copy all apps over but as I said - it can be tricky depending on your push mode.

We use merge_do_default so what we did was gather all the configs from the search heads and simply copied them over to the deployer. With some exceptions. You should not overwrite default on built-in apps (like search) so any changes our users did to the search app we simply migrated to a new app.

The users - if I remember correctly, you can just copy over.

0 Karma

dm1
Contributor

Thank you so much for sharing your valuable experience.

In the case of users, how did you ensure config belonging to the same user residing on two different standalone SHs is migrated successfully without losing any data ?

0 Karma

PickleRick
Ultra Champion

Our case was slightly different (we were simply creating a new SHC based on an existing one). But we simply copied over all users' directories from one SH from old cluster to all SH-s in the hew cluster if I recall correctly

0 Karma
Get Updates on the Splunk Community!

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...