Splunk Enterprise

App Modifications on Search Head Cluster

cchange
Path Finder

My app resides on all search head cluster. I need to modify default files and place in local directory and push across all search heads. I'm placing my app on shcluster folder on deployment server and issued "splunk apply shcluster-bundle -target ". But new files were not placed into the local directory. Can anyone let me know where I need to place the files to update across all the search heads.

Tags (1)
0 Karma

koshyk
Super Champion

SH clustering is bit complex (especially if you have Enterprise Security)
The three elements to do properly a SH cluster are

  1. Staging Server - this is a standalone system whereby you produce all the code and make your changes
  2. Deployer - This can be same as Staging SErver, but the location of the apps should be in "etc/shcluster/apps" and will push to all sh-members
  3. SH members - The final location of apps

So when you create a new app yourself, I tend to put the code in "local". So the app you create , you create in "Staging Server". Pass to deployer and deploy to SH-members. But when it comes to SH-members, it will appear into "default" , no matter you make the changes in "local" in your staging-server. (This is a pain for version-control though). Ensure you pass "-preserve_lookups true" while deploying from deployer, so that the lookups in the SH-members are not overwritten.

So SH-cluster flow in large environments is for your myapp would be:

Staging Server ($SPLUNK_HOME/etc/apps/myapp/) -> deployer ($SPLUNK_HOME/etc/shcluster/apps/myapp) -> SH-members ( $SPLUNK_HOME/etc/apps/myapp)

Going into the file level configurations # I agree this is BAD, but this is how Splunk works

Staging Server ($SPLUNK_HOME/etc/apps/myapp/local/server.conf ) => deployer ($SPLUNK_HOME/etc/shcluster/apps/myapp) => SH-members ($SPLUNK_HOME/etc/apps/myapp/default/server.conf

nickhills
Ultra Champion

When you are "pushing" a new app, in an ideal world you don't want anything other than a 'default' folder in your app.

When an app is installed or upgraded, ONLY the default folder is changed (*). Therefore any files in the local folder on your SHC will remain as they are.

What you may wish to do is collect all the files that have been changed and are now in your SHC/local folders and merge them into the new default folder on the deployer before you push the changes.

  • () Ok, thats not quite true as all the other folders like ./bin ./lookups etc are too, but I was highlighting that specifically the ./local folder is not changed
If my comment helps, please give it a thumbs up!
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...