Forwarders managment after upgrade Splunk enterprise form 9.3.1 to 9.4.2

[heres1]

i have upgrade Splunk enterprise 9.3.1 to 94.2, already restore /etc, but now forwarder managment dose not show any universal phoning home

[livehybrid]

Hi @heres1 

After a Splunk Enterprise upgrade, if Forwarder Management is not showing any "phoning home" (i.e., connected) Universal Forwarders, you probably want to check a few things as below:

• Check that the deployment server (Forwarder Management) settings, SSL certificates, and the deploymentclient configuration on your Universal Forwarders are intact and not overwritten by the upgrade. 
• You mentioned restoring the /etc folder I assume this includes the Splunk Secret in etc/auth ?
• Ensure the deployment server port (default 8089) is up and listening, and network connectivity from forwarders to this port is working. Its worth using curl where possible from one of the UF's to verify this.
• Check $SPLUNK_HOME/var/log/splunk/splunkd.log on both the server and forwarders for phoning home errors.

Upgrades may overwrite configuration files or change SSL settings. If /etc was restored, verify deployment-specific files like deploymentclient.conf (on forwarders) and serverclass.conf (on the deployment server) are correct and certificates/keys are valid. 

Did you just upgrade the Deployment Server, or the UFs too? 

As @kiran_panchavat mentioned - there were changed in 9.2 which affect the indexes used for DS data, although you were already on 9.3.1, right? Were the clients definately showing in Forwarder Management / Agent Manager prior to the upgrade?

Note: The index configuration changes (https://docs.splunk.com/Documentation/Splunk/latest/Updating/Upgradepre-9.2deploymentservers) do not affect the operation of DS, ie it will still deploy apps to the UFs, they just do not show up in the UI, so its worth confirming that they are still able to access the DS!

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[heres1]

Thanks for your answer, i will take into considerations, however i have rollback my upgrading. 

[kiran_panchavat]

@heres1 

Check this 

https://docs.splunk.com/Documentation/Splunk/9.4.2/Updating/Upgradepre-9.2deploymentservers 

This problem can occur in Splunk Enterprise 9.2 or higher if your deployment server forwards its internal logs to a standalone indexer or to the peer nodes of an indexer cluster. This issue can occur after an upgrade or in a new installation of 9.2 or higher. To rectify, add these settings to outputs.conf on the deployment server:

[indexAndForward]
index = true
selectiveIndexing = true     

If you add these settings post-upgrade or post-installation, you might need to restart the deployment server.

Indexers require new internal deployment server indexes

The deployment server uses several internal indexes new in version 9.2. These indexes are included in all indexers at the 9.2 level and higher, but if you try to forward data from those indexes to a pre-9.2 indexer, problems can result.

If you forward data to your indexer tier, create these new internal deployment server indexes in indexes.conf on any pre-9.2 indexers in your environment:

[_dsphonehome]
[_dsclient]
[_dsappevent]

If the indexers are at version 9.2 or higher, they are already configured with those indexes.

Data does not appear when forwarded through an intermediate forwarder

This problem can occur if your deployment server forwards its internal index data through an intermediate forwarder to a standalone indexer or to the peer nodes of an indexer cluster. To rectify, add this setting to outputs.conf on the intermediate forwarder:

[tcpout] 
forwardedindex.2.whitelist = (_audit|_internal|_introspection|_telemetry|_metrics|_metrics_rollup|_configtracker|_dsclient|_dsphonehome|_dsappevent)

If you specify the configuration within a deployment app and use the deployment server to deploy the app to the affected intermediate forwarders, you can later uninstall the app when the intermediate forwarders are upgraded to a future release that incorporates the update.

Deployment Server's Forwarder Management UI exhibits unexpected behaviours after upgrading to versio...

https://community.splunk.com/t5/Splunk-Enterprise/After-upgrading-my-DS-to-Enterprise-9-2-2-clients-...