Clusters and Apps

[oestreicher]

Hi,

I am looking into the the new index replication feature.
I read the documentation and what I understand is that I cannot use the deployment server feature to distribute configuration files to the cluster peers but to use the cluster master instead.
Also I learned that there is no configuration file merging like it is done in the /etc/apps directory. I need to merge index.conf, props.conf, etc. into single files into the etc/master-apps/cluster/local directory.

What I do not understand at the moment is how I handle app installations on a cluster. For example the commonly used technology adapters which need to be installed on the indexing host.

Should I manually merge all index-time configurations from each TA into the single file in the /master-apps/cluster/local directory on the cluster master?
Should I use heavy forwarders in front of my cluster and install the TA Apps there with only merging the index.conf on the cluster master?

Thanks
Dominik

[Vishal_Patel · Accepted solution]

In version 5.0.2 (not released yet but we're working hard to make it available as soon as possible), the master will be able to push multiple apps that contain index time configuration files. Without getting into too much details here, this should alleviate some of your problems around having to merge indexes/props, etc... from multiple apps into the cluster app. The documentation for 5.0.2 release will describe this in more detail.

Although flattening indexes.conf into a single conf file will work, it is a manual, error-prone approach. If at all possible, we recommend that you wait for 5.0.2 release.

[dmlee]

Hi, we do test using version 5.0.2 , master node still won't push other apps (under @SPLUNK_HOME/etc/master-apps/) to index nodes , only _cluster app will be pushed to index nodes.

according to http://docs.splunk.com/Documentation/Splunk/5.0.2/Indexer/Updatepeerconfigurations , it should work ,we will do more test later.

[Lucas_K]

'will be able to push multiple apps that contain index time configuration files'

That sounds like that is only indexer based configurations?

I was hoping there is a way to dictate what app goes to what machines in the same method as deployment server's serverclass.conf.

ie. We REALLY don't need indexing configuration apps being pushed to search heads.

[Lucas_K]

Manual merges? I believe that is what you have to do.

As for TA apps on HF's it all depends. Are they just forwarding events or actually cooking the data via local processing? If it's for former, then your props + transforms will need to be merged with master-apps also. If it's the later, then you should only need to copy your indexes.conf into your master apps. The local app will remain the same.

Hopefully someone from splunk responds here as I am interested to hear the response about the best way to do this.

We are currently trying to figure out a way to migrate from a nicely configured distribution server in 4.x to 5.x.

From my own personally testing I have noticed the following (and its quite possibly wrong also!).

No other apps apart from master-apps/cluster get replicated around via the master cluster server (which you've already seen). Also this new master-apps/cluster becomes slave-apps/cluster on the actual index. This app now becomes the top level in the new 5.0.x cluster precedence order ( http://docs.splunk.com/Documentation/Splunk/5.0.1/admin/Wheretofindtheconfigurationfiles ) that is applied to the indexes existing configuration.

The following all depends on how your currently distributing your apps.

In my environments we have a configuration layout like this :

• input apps - exists only on UF's.
• parsing apps - exists on indexes or heavy forwarders (as required).
• search apps - exists on search heads only.

To migrate to a clustered version of this I see the following occuring :

• input apps - exists only on UF's. No change. Still use deployment server
• search apps - exists on search heads only. No change. Still use deployment server.
• parsing apps - exists on indexes or heavy forwarders (as required). Manually copy and merge all of these configs into a single master-apps/cluster/local configuration. No dist server able to be used as the built in cluster replication takes over this particular dir location. Manual updating to the master-apps file location on the cluster master is required. In addition to a manual command to trigger the update.

This last section could possibly be scripted so that it gets built from existing repositories that exist under the deployment-apps folder structure. Not entirely pretty.