All Apps and Add-ons

Why do edits to inputs.conf for Splunk_TA_windows on the deployment server result in "app=Splunk_TA_windows was already installed...it may not be overridden via deployment server"?

k2skaterii
Path Finder

I just set up my first Splunk Deployment server. I'm trying to get used to how it works, and how to manage it. In an attempt to K.I.S.S. I decided that my first app that I would deploy and manage would be the the "Windows Universal Forwarder" AKA "Splunk_TA_Windows".

After deploying the app from my deployment server to my clients, I attempted to modify the inputs.conf on the deployment server with the expectation that the change would propagate down to all the clients. Unfortunately that did not happen. After successfully deploying the app to the six clients in my "Test" server class, the updated inputs.conf did not propagate down to the six clients.

After editing the .\etc\Splunk_TA_Windows\local\inputs.conf I believe I reloaded the deployment server properly with the following command: .\bin\splunk reload deploy-server.

Now my clients are all showing the same entries in the splunkd.log

  • Successful App deployment:

    10-21-2015 12:34:29.626 -0500 INFO DeployedApplication - Installing app=Splunk_TA_windows to='D:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows'

  • Failed App update:

    10-21-2015 12:41:51.332 -0500 INFO DeployedApplication - Checksum mismatch 0 <> 1316440081260235894 for app=Splunk_TA_windows. Will reload from=':8090/services/streams/deployment?name=default:Test%203:Splunk_TA_windows'

    10-21-2015 12:41:51.752 -0500 INFO DeployedApplication - Downloaded url=:8090/services/streams/deployment?name=default:Test%203:Splunk_TA_windows to file='D:\Program Files\SplunkUniversalForwarder\var\run\Test 3\Splunk_TA_windows-1445449223.bundle' sizeKB=3500

    10-21-2015 12:41:52.591 -0500 WARN DeployedApplication - app=Splunk_TA_windows, installed_via="search head cluster deployer, UI, CLI, or REST API", checksum=0a01c29a8ea0ff2831f002c02fe365f9210ad2d7

    10-21-2015 12:41:52.591 -0500 WARN DeployedApplication - app=Splunk_TA_windows was already installed via search head cluster deployer, UI, CLI, or REST API; it may not be overridden via deployment server; remove existing app=Splunk_TA_windows via search head cluster deployer, UI, CLI, or REST API if you wish to install it via deployment server

    10-21-2015 12:41:52.591 -0500 ERROR DeployedServerclass - name=Test 3 Failed to install app=Splunk_TA_windows

To troubleshoot this I've:
- Restarted the SplunkUniversalForwarder service on the affected client. No change.
- Restarted the Deployment server. Its a Windows VM, so that should have been trouble shooting step number 1. No change.
- Manually removed the app folder from the client, and then restarted the SplunkUniversalForwarder service. After doing this the App will deploy successfully, but the problem reoccurs if I try to edit the inputs.conf file

What am I missing?

What did I do wrong when I set up my deployment server?

Thanks for the help!
Ken

1 Solution

jcrabb_splunk
Splunk Employee
Splunk Employee

We've filed a bug for this issue, SPL-108220. It has now been added to the known issues for both 6.3.0 and 6.3.1.


SPL-108220: Unable to deploy an app through Deployment Server Forwarder Management. Error: app= was already installed via search head cluster deployer, UI, CLI, or REST API; it may not be overridden via deployment server; remove existing app= via search head cluster deployer, UI, CLI, or REST API if you wish to install it via deployment server.


The way Splunk detects SHC "Deployer" managed apps (or generally: apps installed via UI, CLI, or REST API) in the client is by the presence of a setting, install_source_checksum in app.conf. If an app which was deployed by a Deployment Servers Forwarder Management contains an app.conf file with an install_source_checksum setting, then the reported error will occur and the app cannot be updated. One way this could happen is if an app was installed through the web gui on a Search Head and then copied to the deployment_apps repository and subsequently pushed to clients. When installing through the gui, it writes this checksum to the configuration file. Here is an example of how that looks:

$SPLUNK_HOME/etc/app//default/app.conf:

[install]
install_source_checksum = 0f98de8a93964bc7d52e79c7fb834b67ed739eaz

If that setting is removed prior to deploying the app the first time, the issue can be avoided. However, if you have already deployed the app, I don't believe it can be corrected unless this setting is removed from each client manually and the originating app on the Deployment Server. This should be addressed in a future release, currently targeted for 6.3.2.

Jacob
Sr. Technical Support Engineer

View solution in original post

mikaelbje
Motivator

As a workaround I deployed an app with a Windows batch script called by a scripted input which deleted app.conf from Splunk_TA_windows on the forwarders.

This fixed the problem on all my forwarders.

After about five minutes when they had all executed the script I undeployed the app to stop it from removing the correct app.conf that was deployed.

Bear in mind that you also need to remove the install_source_checksum line from the Splunk_TA_windows/default/app.conf file in the deployment-apps directory on the deployment server as pointed out above.

napomokoetle
Communicator

Excellent mikaelbje. You rock!

0 Karma

jcrabb_splunk
Splunk Employee
Splunk Employee

We've filed a bug for this issue, SPL-108220. It has now been added to the known issues for both 6.3.0 and 6.3.1.


SPL-108220: Unable to deploy an app through Deployment Server Forwarder Management. Error: app= was already installed via search head cluster deployer, UI, CLI, or REST API; it may not be overridden via deployment server; remove existing app= via search head cluster deployer, UI, CLI, or REST API if you wish to install it via deployment server.


The way Splunk detects SHC "Deployer" managed apps (or generally: apps installed via UI, CLI, or REST API) in the client is by the presence of a setting, install_source_checksum in app.conf. If an app which was deployed by a Deployment Servers Forwarder Management contains an app.conf file with an install_source_checksum setting, then the reported error will occur and the app cannot be updated. One way this could happen is if an app was installed through the web gui on a Search Head and then copied to the deployment_apps repository and subsequently pushed to clients. When installing through the gui, it writes this checksum to the configuration file. Here is an example of how that looks:

$SPLUNK_HOME/etc/app//default/app.conf:

[install]
install_source_checksum = 0f98de8a93964bc7d52e79c7fb834b67ed739eaz

If that setting is removed prior to deploying the app the first time, the issue can be avoided. However, if you have already deployed the app, I don't believe it can be corrected unless this setting is removed from each client manually and the originating app on the Deployment Server. This should be addressed in a future release, currently targeted for 6.3.2.

Jacob
Sr. Technical Support Engineer

k2skaterii
Path Finder

In my case there may have been a slight P.E.B.C.A.C. issue. Since this was my the first App I deployed through the deployment server, i know i followed some instructions, but I may have followed the instructions for the wrong scenario.

Looking back I think I installed the downloaded App, through the deployment server's web GUI. Then After it was installed I copy pasted the folder from the $Splunk\etc\apps folder to the $Splunk\etc\deployment-apps folder. Then modified the inputs.conf and THEN proceeded to deploy the App out to all my clients.

It looks like I'll have to delete the line from the app.conf then delete the Splunk_TA_Windows folder from all my forwarders, and allow the folder to re-download. Once that's done, hopefully things will just "just work".

Thanks for the response.

0 Karma

napomokoetle
Communicator

Work around works but is not feasible for me since I don't access to the agent hosts to be able to manually edit the app.conf! 😞
Any estimates as to when might expect to be able to download Splunk 6.3.2?

0 Karma

denisd
New Member

It worked. I had to delete the line on each client too.

0 Karma

napomokoetle
Communicator

I'm experiencing the same exact problem. Stuck!
I don't have a search head cluster so I couldn't have done what the log states!!! I have two search heads but they are NOT clustered. Moreover, I've NOT used them to distribute applications!

10-27-2015 13:31:32.682 +0200 WARN DeployedApplication - app=Splunk_TA_windows was already installed via search head cluster deployer, UI, CLI, or RES
T API; it may not be overridden via deployment server; remove existing app=Splunk_TA_windows via search head cluster deployer, UI, CLI, or REST API if
you wish to install it via deployment server

0 Karma

t9445
Path Finder

We have the same situation - what I ended up doing (short-term fix) was to write another app - that disables the Splunk_TA_windows app via an update to its local/app.conf, then disables itself as well), since we have our own collections of the Windows eventlogs, we cannot use the ones enabled in the Splunk_TA_windows app, so we had to turn it off. As new UFs are enabled they automatically get the "disable-app", unfortunately there is still a stream of several windows events that are lost until the Splunk_TA_windows app is disabled at initial startup.

las
Contributor

Would it be possible for you, to share your app?
That way others might benefit, and won't have to start from scratch.
kind regards

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...