Getting Data In

Why is the Splunk universal forwarder dropping from deployment server?

rameshtdp
New Member

Splunk UF's are having different versions 6.0.0, 6.3 and 6.5.2 are connecting to Deployment server with 7.2.6 server.
All of sudden some of the clients are dropping from Deployment server.

0 Karma
1 Solution

mguhad
Communicator

Hi,
i have seen this happen before where the deploymentclient.conf is initially place under /etc/apps/local/deploymentclient.conf
and then once the client has established connection with deployment server (phones home), the same app is configured on the deployment server and pushed out to client which overrides the initial app () and this is what causes the breaking as splunk warns against this in below documentation snippet:
https://docs.splunk.com/Documentation/Splunk/7.2.6/Updating/Configuredeploymentclients

Important: Do not use the deployment server to push deploymentclient.conf updates to the deployment clients. Doing so can potentially lead to situations where the deployment clients lose their ability to contact the deployment server.

You should also check for this kind of error as splunk tries to stack the app containing the deploymentclient.conf configs on top of each other due to the app being wrongly pushed out via deployment sever as warned against above:

ERROR DeployedApplication - untarToDest, failed to mkdir opt/splunkforwarder/etc/apps/<my_app>/<my_app>/<my_app>/<my_app>/<my_app>/local: Permission denied

To fix this issue, What we did we removed the app with deploymentclient.conf configs from the deployment server and re-configured the deployment client app again on the remote server or ran ./splunk set deploy-poll <IP_address/hostname>:<management_port>

View solution in original post

0 Karma

mguhad
Communicator

Hi,
i have seen this happen before where the deploymentclient.conf is initially place under /etc/apps/local/deploymentclient.conf
and then once the client has established connection with deployment server (phones home), the same app is configured on the deployment server and pushed out to client which overrides the initial app () and this is what causes the breaking as splunk warns against this in below documentation snippet:
https://docs.splunk.com/Documentation/Splunk/7.2.6/Updating/Configuredeploymentclients

Important: Do not use the deployment server to push deploymentclient.conf updates to the deployment clients. Doing so can potentially lead to situations where the deployment clients lose their ability to contact the deployment server.

You should also check for this kind of error as splunk tries to stack the app containing the deploymentclient.conf configs on top of each other due to the app being wrongly pushed out via deployment sever as warned against above:

ERROR DeployedApplication - untarToDest, failed to mkdir opt/splunkforwarder/etc/apps/<my_app>/<my_app>/<my_app>/<my_app>/<my_app>/local: Permission denied

To fix this issue, What we did we removed the app with deploymentclient.conf configs from the deployment server and re-configured the deployment client app again on the remote server or ran ./splunk set deploy-poll <IP_address/hostname>:<management_port>

0 Karma

rameshtdp
New Member

Thanks mguhad!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...