App Modifications on Search Head Cluster

[cchange]

My app resides on all search head cluster. I need to modify default files and place in local directory and push across all search heads. I'm placing my app on shcluster folder on deployment server and issued "splunk apply shcluster-bundle -target ". But new files were not placed into the local directory. Can anyone let me know where I need to place the files to update across all the search heads.

[koshyk]

SH clustering is bit complex (especially if you have Enterprise Security)
The three elements to do properly a SH cluster are

1. Staging Server - this is a standalone system whereby you produce all the code and make your changes
2. Deployer - This can be same as Staging SErver, but the location of the apps should be in "etc/shcluster/apps" and will push to all sh-members
3. SH members - The final location of apps

So when you create a new app yourself, I tend to put the code in "local". So the app you create , you create in "Staging Server". Pass to deployer and deploy to SH-members. But when it comes to SH-members, it will appear into "default" , no matter you make the changes in "local" in your staging-server. (This is a pain for version-control though). Ensure you pass "-preserve_lookups true" while deploying from deployer, so that the lookups in the SH-members are not overwritten.

So SH-cluster flow in large environments is for your myapp would be:

Staging Server ($SPLUNK_HOME/etc/apps/myapp/) -> deployer ($SPLUNK_HOME/etc/shcluster/apps/myapp) -> SH-members ( $SPLUNK_HOME/etc/apps/myapp)

Going into the file level configurations # I agree this is BAD, but this is how Splunk works

Staging Server ($SPLUNK_HOME/etc/apps/myapp/local/server.conf ) => deployer ($SPLUNK_HOME/etc/shcluster/apps/myapp) => SH-members ($SPLUNK_HOME/etc/apps/myapp/default/server.conf

[nickhills]

When you are "pushing" a new app, in an ideal world you don't want anything other than a 'default' folder in your app.

When an app is installed or upgraded, ONLY the default folder is changed (*). Therefore any files in the local folder on your SHC will remain as they are.

What you may wish to do is collect all the files that have been changed and are now in your SHC/local folders and merge them into the new default folder on the deployer before you push the changes.

• () Ok, thats not quite true as all the other folders like ./bin ./lookups etc are too, but I was highlighting that specifically the ./local folder is not changed