Deployment Architecture

How do I replicate settings in system/local across the search head cluster?

chustar
Path Finder

When using a stand alone search head, we made configuration changes in etc/system/local/e.g. outputs.conf, limits.conf, etc

I've converted this standalone instance to a search head cluster, but I don't want to go into each cluster member and reconfigure these settings.

How would I ensure that I can create the configurations in one place and replicate them to the cluster members?

My current idea is to add these configurations to the deployer e.g. etc/shcluster/apps/custom_configs/limits.conf and then set the app to export its settings using export=system.
This worked when migrating savedsearches and custom apps, but I worry that the same is not true for configurations that are not part of any app.

0 Karma
1 Solution

beatus
Communicator

Chustar,
You're correct that you need to deploy them with the deployer. As you said, a "custom_configs" app is perfect for this.

I do see some folks splitting these custom configs up into a few apps, such as:

  • Indexes (for autocomplete). Often there is an indexes app that can be copied from the Indexer Cluster Master Node and re-used. Just remember to adjust the volume configurations or make them present.
  • Outputs app. Often there is an outputs app already configured correctly for Universal Forwarders in the environment on the Deployment Server.
  • A "SHC_settings" app for anything left over. limits.conf, server.conf, alert_actions.conf, etc. Whatever you need to do here that's SHC or SH specific.

There's no requirement to split them out into apps, but if you already have apps that do most of the things you'd like to push to the SHC then it makes sense to reuse them. If you prefer a combined app, that's fine too.

View solution in original post

0 Karma

beatus
Communicator

Chustar,
You're correct that you need to deploy them with the deployer. As you said, a "custom_configs" app is perfect for this.

I do see some folks splitting these custom configs up into a few apps, such as:

  • Indexes (for autocomplete). Often there is an indexes app that can be copied from the Indexer Cluster Master Node and re-used. Just remember to adjust the volume configurations or make them present.
  • Outputs app. Often there is an outputs app already configured correctly for Universal Forwarders in the environment on the Deployment Server.
  • A "SHC_settings" app for anything left over. limits.conf, server.conf, alert_actions.conf, etc. Whatever you need to do here that's SHC or SH specific.

There's no requirement to split them out into apps, but if you already have apps that do most of the things you'd like to push to the SHC then it makes sense to reuse them. If you prefer a combined app, that's fine too.

0 Karma
Get Updates on the Splunk Community!

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...