Getting Data In

Why is the Splunk universal forwarder dropping from deployment server?

rameshtdp
New Member

Splunk UF's are having different versions 6.0.0, 6.3 and 6.5.2 are connecting to Deployment server with 7.2.6 server.
All of sudden some of the clients are dropping from Deployment server.

0 Karma
1 Solution

mguhad
Communicator

Hi,
i have seen this happen before where the deploymentclient.conf is initially place under /etc/apps/local/deploymentclient.conf
and then once the client has established connection with deployment server (phones home), the same app is configured on the deployment server and pushed out to client which overrides the initial app () and this is what causes the breaking as splunk warns against this in below documentation snippet:
https://docs.splunk.com/Documentation/Splunk/7.2.6/Updating/Configuredeploymentclients

Important: Do not use the deployment server to push deploymentclient.conf updates to the deployment clients. Doing so can potentially lead to situations where the deployment clients lose their ability to contact the deployment server.

You should also check for this kind of error as splunk tries to stack the app containing the deploymentclient.conf configs on top of each other due to the app being wrongly pushed out via deployment sever as warned against above:

ERROR DeployedApplication - untarToDest, failed to mkdir opt/splunkforwarder/etc/apps/<my_app>/<my_app>/<my_app>/<my_app>/<my_app>/local: Permission denied

To fix this issue, What we did we removed the app with deploymentclient.conf configs from the deployment server and re-configured the deployment client app again on the remote server or ran ./splunk set deploy-poll <IP_address/hostname>:<management_port>

View solution in original post

0 Karma

mguhad
Communicator

Hi,
i have seen this happen before where the deploymentclient.conf is initially place under /etc/apps/local/deploymentclient.conf
and then once the client has established connection with deployment server (phones home), the same app is configured on the deployment server and pushed out to client which overrides the initial app () and this is what causes the breaking as splunk warns against this in below documentation snippet:
https://docs.splunk.com/Documentation/Splunk/7.2.6/Updating/Configuredeploymentclients

Important: Do not use the deployment server to push deploymentclient.conf updates to the deployment clients. Doing so can potentially lead to situations where the deployment clients lose their ability to contact the deployment server.

You should also check for this kind of error as splunk tries to stack the app containing the deploymentclient.conf configs on top of each other due to the app being wrongly pushed out via deployment sever as warned against above:

ERROR DeployedApplication - untarToDest, failed to mkdir opt/splunkforwarder/etc/apps/<my_app>/<my_app>/<my_app>/<my_app>/<my_app>/local: Permission denied

To fix this issue, What we did we removed the app with deploymentclient.conf configs from the deployment server and re-configured the deployment client app again on the remote server or ran ./splunk set deploy-poll <IP_address/hostname>:<management_port>

0 Karma

rameshtdp
New Member

Thanks mguhad!

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...