Deployment server does not forward app to HF

[SplunkExplorer]

Hi Splunkers, I'm deploying a new Splunk Enterprise environment; inside it, I have (for now) 2 HF and a DS.
I'm trying to set an outputs.conf file on both HF via DS; clients perform a correct phoning to DS, but then apps are not downloaded. I checked the internal logs and I got no error related to app.
I followed doc and course material used during Architect course for references.

Below, configuration I made on DS.

App name: 

/opt/splunk/etc/deployment-apps/hf_seu_outputs/

App file

/opt/splunk/etc/deployment-apps/hf_seu_outputs/default/app.conf

[ui]
is_visible = 0

[package]
id = hf_outputs
check_for_updates = 0

/opt/splunk/etc/deployment-apps/hf_seu_outputs/local/outputs.conf

[indexAndForward]
index=false

[tcpout]
defaultGroup = default-autolb-group
forwardedindex.filter.disable = true
indexAndForward = false

[tcpout:default-autolb-group]
server=<idx1_ip_address>:9997, <idx2_ip_address>:9997, <idx3_ip_address>:9997

serverclass.conf:

[serverClass:spoke_hf:app:hf_seu_outputs]
restartSplunkWeb = 0
restartSplunkd = 1
stateOnClient = enabled

[serverClass:spoke_hf]
whitelist.0 = <HF1_ip_address>, <HF1_ip_address>

File and folder permission are right, owner is the user used to execute Splunk (in a nutshell, the owner of /opt/spluk). I suppose it is a very stupid issue, but I'm not able to figured it out.

[SplunkExplorer · Accepted solution]

Solved.

The issue was related to this configuration sgement:

[serverClass:spoke_hf]
whitelist.0 = <HF1_ip_address>, <HF1_ip_address>

Searching better into documentation, I found that comma separated IP address list is not supported.
After I switched to:

whitelist.0 = <HF1 IP Address>
whitelist.1 = <HF2 IP Address>

Issue went away.

[deepakc]

A few things to check - (I know you have done some already)  

Check that your serverclass is taking the current config (might be some config that’s  overriding, its normally in /opt/splunk/etc/system/local/serverclass and sometimes in a dedicated app /opt/splunk/bin/splunk btool serverclass list --debug

Check the Permissions on the HF's /opt/splunk/etc/apps/  (sudo chown -R splunk:splunk /opt/splunk/etc/apps - this is typical)

Restart the HF / Deployment Server

Can you verify the ownership of the apps on the Deployment Server (Typically they should be splunk:splunk sudo chown -R splunk:splunk /opt/splunk/etc/deployment_apps)

Can you verify the firewall ports are all OK 8089 (HF to DS - port 8089)

Can you double check the apps names in serverclass.conf (I have seen app name typo's errors in the past)  

[SplunkExplorer]

Hi @deepakc, following output of required checks:

Check that your serverclass is taking the current config (might be some config that’s  overriding, its normally in /opt/splunk/etc/system/local/serverclass and sometimes in a dedicated app /opt/splunk/bin/splunk btool serverclass list --debug - Done: the only 2 serverclass.conf files are the ones under $SPLUNK_HOME$/etc/system/default and $SPLUNK_HOME$/etc/system/local

Check the Permissions on the HF's /opt/splunk/etc/apps/  (sudo chown -R splunk:splunk /opt/splunk/etc/apps - this is typical) - Done, folder ownership is fine

Restart the HF / Deployment Server - Done

Can you verify the ownership of the apps on the Deployment Server (Typically they should be splunk:splunk sudo chown -R splunk:splunk /opt/splunk/etc/deployment_apps) - Done, ownership if fine

Can you verify the firewall ports are all OK 8089 (HF to DS - port 8089) - Done, HFs can reach DS on 8089 and vice versa

Can you double check the apps names in serverclass.conf (I have seen app name typo's errors in the past)  - Done, app folder name and app name in serveclass.conf are the same

[deepakc]

Hi @SplunkExplorer 
Can you check on the HF's /opt/splunk/etc/apps folder if there are some outputs apps there (Left overs perhaps from testing etc) if so remove the app into a /tmp folder, restart HF's,  and push via the deployment server only.