Getting Data In

## Installing a universal forwarder in low privilege mode, why am I getting error "Deployment Server not available on a dedicated forwarder"?

Motivator

Our admin created me a regular domain user to test low P and assigned it these privileges:

``````• Permission to log on as a service.
• Permission to log on as a batch job.
• Permission to replace a process-level token.
• Permission to act as part of the operating system.
• Permission to bypass traverse checking
``````

I run this to test the automation:

``````msiexec /i splunkforwarder-6.3.2-aaff59bb082c-x64-release.msi AGREETOLICENSE=Yes INSTALLDIR=c:\SplunkUniversalForwarder RECEIVING_INDEXER=heavy.forwarder:9997 DEPLOYMENT_SERVER=deploy.server:8089 SET_ADMIN_USER=0 LOGON_USERNAME=DOMAIN\splunklpuser LOGON_PASSWORD=somethingclever /quiet /log  lar.txt
``````

The lar.txt log shows a 1603 permissions error and the `appdata\local\temp\splunk.log` shows this as the failure point:

``````Deployment Server not available on a dedicated forwarder
``````

The communication path to the deployment server is open and if I install with LocalSystem, then it is successful.

What is my `DOMAIN\splunklpuser` userid missing?

1 Solution
Motivator

It is definitely the super complex password with special charaters.

I resolved it in Powershell by escaping the entire password in single quotes.

``````LOGON_PASSWORD='somethingclever'
``````

cmd.exe still does not like one of the special characters in the password, but we should be able to escape it with a carat.

It still isn't clear why that particular error message occurred when the problem was something else entirely.

Motivator

It is definitely the super complex password with special charaters.

I resolved it in Powershell by escaping the entire password in single quotes.

``````LOGON_PASSWORD='somethingclever'
``````

cmd.exe still does not like one of the special characters in the password, but we should be able to escape it with a carat.

It still isn't clear why that particular error message occurred when the problem was something else entirely.

Motivator

I'm on to something - not to be confused with "on something" either.

I threw LAUNCHSPLUNK=0 into the mix and tried to start the service manually after the installer completed. That resulted in a 1069 logon failure, so I went into the "Log On" tab on the service properties and pasted in the password I used on the command line. Magic time.

So, something about the totally unreadable, unimaginable and ridiculously unmemorizable password is the problem. I wrapped it in quotes on the command line and that made no difference, but I finally have some evidence to go on.

Path Finder

@lycollicott , thanks for the post. We were having the same issue on a Server 2016 box and using LaunchSplunk=0 resolved it.

SplunkTrust

Check this out too:

https://support.microsoft.com/en-us/kb/834484

Tells the possible causes of the 1603 and how to resolve.

"You may receive this error message if any one of the following conditions is true:
The folder that you are trying to install the Windows Installer package to is encrypted.
The drive that contains the folder that you are trying to install the Windows Installer package to is accessed as a substitute drive.
The SYSTEM account does not have Full Control permissions on the folder that you are trying to install the Windows Installer package to. You notice the error message because the Windows Installer service uses the SYSTEM account to install software."

Motivator

We verified permissions with the admins multiple times and we don't see anything that explains the 1603.

SplunkTrust

So you dont have bitlocker or any other form of encryption?

Motivator

No. I think the 1603 was the parent message of a 1069 when the misinterpreted password was used.

SplunkTrust

What version(s) of windows please? It looks like it cant bind to any ports. I cant find the GPO for port binding to save my life.

Motivator

Windows 2012r2

SplunkTrust

Hi,

Splunk used to discourage setting the deployment server during msiexec / installation on universal forwarders. It appears maybe they no longer allow it???

So here is what you need to do, same command minus the deployment server argument. Then you need to run

`````` c:\splunkuniversalforwarder\splunk.exe set deploy-poll deploymentserverHostOrIp:8089
``````

The docs here say I'm crazy, and maybe I am:

http://docs.splunk.com/Documentation/Splunk/6.3.3/Forwarding/DeployaWindowsdfviathecommandline

Still give it a shot and let me know please.

Motivator

I deployed many UFs remotely with the command line specifying the deployment server and they all worked fine as long as I let LocalSystem run the services. I only have this issue when specifying low P mode.

Influencer

TBH I would recommend opening a support case: http://login.splunk.com/page/sso_redirect?type=portal

Motivator

Oh, i do have a case open too.

SplunkTrust

All right then I'm crazy bat stuff. Did you try the command without specifying the deployment server?

If it works, would it then make sense that you could automate a second command that sets the deployment server? I guess it's a workaround, but it's barely a new line of code.

Splunk Employee

Can you share a couple more log messages from splunkd.log BEFORE the Deployment Server message is issued, please? This message should not appear in isolation, there should be others in the DC:DeploymentClient category.

Motivator
``````processed file: C:\SplunkUniversalForwarder\var\spool\dirmoncache
processed file: C:\SplunkUniversalForwarder\var\spool\splunk
Successfully processed 29 files; Failed processing 0 files
HTTP/1.1 200 OK
Date: Fri, 05 Feb 2016 20:09:06 GMT
Expires: Thu, 26 Oct 1978 00:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, max-age=0
Content-Type: text/xml; charset=UTF-8
X-Content-Type-Options: nosniff
Content-Length: 1930
Connection: Close
X-Frame-Options: SAMEORIGIN
Server: Splunkd

<?xml version="1.0" encoding="UTF-8"?>
<!--This is to override browser formatting; see server.conf[httpServer] to disable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .-->
<?xml-stylesheet type="text/xml" href="/static/atom.xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:s="http://dev.splunk.com/ns/rest" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
<title>localapps</title>
<id>/services/apps/local</id>
<updated>2016-02-05T20:09:06+00:00</updated>
<generator build="aaff59bb082c" version="6.3.2"/>
<author>
<name>Splunk</name>
</author>
<opensearch:totalResults>0</opensearch:totalResults>
<opensearch:itemsPerPage>30</opensearch:itemsPerPage>
<opensearch:startIndex>0</opensearch:startIndex>
<s:messages>
<s:msg type="INFO">Restart required by: default-mode, limits, server, web</s:msg>
</s:messages>
</feed>
DS init failed: Deployment Server not available on a dedicated forwarder.
``````

Splunk Employee

Can you share a couple more log messages from splunkd.log BEFORE the Deployment Server message is issued, please? This message should not appear in isolation, there should be others in the DC:DeploymentClient category.

Influencer

Can it make an outgoing connection to deploy.server:8089 via TCP?

Motivator

Yes it does. That was the first thing I checked.

The Latest From the Splunk Community!