topic Re: how to setup basic deployment for universalforwarder? in Deployment Architecture

how to setup basic deployment for universalforwarder?

kyless — Thu, 08 Nov 2012 21:35:14 GMT

What am I missing? (new to splunk, but have been reading all about deployment servers)

test environment with 2 servers --

Splunk 5 installed as deployment server @ server named x.y.z

Splunk UniversalForwarder 5 installed on client @ server name austest

On Deployment Server:

$SPLUNKHOME/etc/system/local/serverclass.conf

[serverClass:testing]

filterType = whitelist

repositoryLocation = /opt/splunk/etc/deployment-apps/testing/

whitelist.0 = aus*

Placed inputs.conf and outputs.conf at:

$SPLUNKHOME/etc/deployment-apps/testing/default

outputs.conf

[tcpout]

disabled = false

defaultGroup=splunkPOC

[tcpout:splunkPOC]

server=x.y.z:9997

[tcpout-server://x.y.z:9997]

inputs.conf

[monitor:///var/log/messages]

disabled=false

sourcetype=syslog

On client:

$SPLUNKHOME/etc/system/local/deploymentclient.conf

[target-broker:deploymentServer]

targetUri = x.y.z:8089

Enabled the receiver tcp port 9997 on the indexer. (Previously done when testing a 'non-deployment server' setup, which was full functional on forwarding from the client.)

Reloaded deploy-server

/opt/splunk/bin/splunk reload deploy-server

Check that client is configured:

/opt/splunkforwarder/bin/splunk list deploy-poll

Deployment Server URI is set to "x.y.z:8089".

I can see the client reaching the deployment server (at Splunk Web) and via:

(at deployment server)

/opt/splunk/bin/splunk list deploy-clients | grep 'hostname:'

hostname: austest

But the client doesn't appear to retrieve the inputs.conf or outputs.conf

(at client)

/opt/splunkforwarder/bin/splunk list forward-server

Active forwards:

None

Configured but inactive forwards:

None

No data is forwarded from the client.
What is wrong?

Where should the files be located after retrieval from the deployment server?

(Manual configuration works for the client to send data, but obviously isn't scalable.)

Re: how to setup basic deployment for universalforwarder?

sowings — Thu, 08 Nov 2012 21:43:25 GMT

If the app is called 'testing', the repositoryLocation you've specified are wrong. It's expected to be the directory containing the apps (e.g. $SPLUNK_HOME/etc/deployment-apps), not the name of the app itself.

Note that when the app is deployed to the client, it will be deployed to the $SPLUNK_HOME/etc/apps, so you can check the the filesystem for that. You may need to include a metadata/local.meta (to indicate sharing permissions) for the app in question. You may also need an app.conf in the app's local/ subdir.

Finally, changes to inputs.conf typically require a restart, so you won't see that system as a forwarder until the forwarder system has had its Splunk daemon restarted.

Re: how to setup basic deployment for universalforwarder?

kyless — Thu, 08 Nov 2012 22:03:35 GMT

Thanks.

I am not trying to install an app, just basic universalforwarder configuration. Installation of the *Nix app will come later if I can get the basics down.

I have tried restarting the client, with no joy.

Re: how to setup basic deployment for universalforwarder?

sowings — Thu, 08 Nov 2012 22:07:20 GMT

When you said that you created an inputs.conf and outputs.conf in the testing/default directory, you were in fact creating an "app" called testing, that should be sent to the client. The client will request it from the DS; this logs via a facility called PackageDownloadRestHandler. Grep for that in your splunkd.log on the DS.

It sounds like the client doesn't realize that it needs the app, OR the app isn't installing correctly. Did you update repositoryLocation in serverclass.conf, and reload deploy-server?

Re: how to setup basic deployment for universalforwarder?

kyless — Thu, 08 Nov 2012 22:58:57 GMT

Ok. I'm new, thanks for helping get my mind around terminology.

I'm basically following --
http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Extendedexampledeployseveralstandardforwarders

I had used Splunk Web to create the serverclass configuration and it required a 'repository location'. I've now changed the serverclass.conf to reflect the base repository --

repositoryLocation = $SPLUNK_HOME/etc/deployment-apps

restarted Splunk on the DS
restarted Splunk on the client

Grepping for 'PackageDownload' doesn't show up on the DS logs, historical or new.

Re: how to setup basic deployment for universalforwarder?

sowings — Fri, 09 Nov 2012 03:42:10 GMT

Are you literally doing 'grep' from the command line, or using Splunk search? Splunk search won't find PackageDownload by itself, because that term doesn't exist "in isolation"; you'd have to search for the full PackageDownloadRestHandler.

But I'm going to guess that this client just doesn't know it's supposed to get the app. Does the host show up in splunk list deploy-clients?

Re: how to setup basic deployment for universalforwarder?

kyless — Fri, 09 Nov 2012 17:06:55 GMT

Yes. Using grep from command line at both the DS and client against:
$SPLUNK_HOME/var/log/splunk/*

Yes. Client name appears on the DS when I run 'splunk list deploy-clients'.

Re: how to setup basic deployment for universalforwarder?

sowings — Fri, 09 Nov 2012 17:58:12 GMT

Ok, so the client is phoning home (good), but not realizing that it has to download the app, since you're not seeing it in PackageDownloadRestHandler.

If the snippet of serverclass.conf you're provided above is the whole thing, you may be missing an app declaration (i.e., send this app to servers in this class).

Consider adding [serverClass:testing:app:testing] to your serverclass.conf and reload the deployment server.

Re: how to setup basic deployment for universalforwarder?

sowings — Tue, 13 Nov 2012 21:45:27 GMT

Were you ultimately able to get this working?

Re: how to setup basic deployment for universalforwarder?

kyless — Wed, 14 Nov 2012 21:17:29 GMT

Sorry for the delay. Other items took priority.

Adding in the [serverClass:testing:app:testing] to the serverclass.conf worked.