Architecture: 3 search-heads in a cluster, captain dynamically elected, 1 search head deployer that is not one of the search head servers.
I go to $SPLUNK_HOME/etc/shcluster/apps/, create my app, add local/data/ui/{nav/default.xml,views/(empty)} that is valid (per https://dev.splunk.com/enterprise/docs/developapps/createapps/addnavsplunkapp/)
Push type is merge_to_default - so when I run splunk apply shcluster-bundle command, it should merge the local folder into default and deploy that. That doesn't happen, however.
When I check the app on SearchHeads I see that the app is deployed, but there are only two folders inside: default with app.conf:
[ui]
is_visible = 1
show_in_nav = 1
label = Testing Label
[launcher]
author = myname
description = Some label
version = 1.0
[install]
install_source_checksum = <digest>
The second folder metadata contains single file metadata.local with nothing particularly interesting (except version that I am running on - 8.0.2). Note default.xml (or any other UI stuff) is nowhere to be found.
When I open the app on the search heads UI, I can select it from navigation and I am automatically redirected to Appname/Inputs despite this is not defined anywhere in my default.xml that I created on SHD.
I dig a little bit and found that /Inputs are defined in Splunk for AWS Addon's default.xml that is apparently being used for the application. I can also make my search head app to use my default.xml if I manually create local folder on each of the search heads - but I don't find this approach good from the maintenance perspective.
My questions:
1: Where should I put my default.xml on SH deployer for it to be deployed and used on Search Heads?
2: Why does SearchHead automatically assigns default.xml from different app? Can I configure this somewhere? Can I maybe create a new default.xml to be selected to be used for this app (and future apps I create)?
Thank you for your time!