Introducing Deployment Server Stops Data Getting In

[achauhan2098]

Hi Community! 

Despite lots of reading and doing my best to get the answer from documentation, I can't see why the introduction of a deployment server is causing issues with the data getting into Splunk Cloud. 

So I'd really appreciate some help.

I have 4 test servers and I've completed the steps below: 

• Intermediate forwarders have Splunk cloud as forwarding servers in outputs.conf file. this is also verified when issuing cli commands.
• Intermediate forwarders have a receiving port configured
• Intermediate forwarders have the Splunk Cloud credentials installed 
• When the UF have the intermediate forwarders set as the forwarding server the cli shows that this config is good

-> and ultimately the forwarding client data reaches the cloud index no problem. 

However when I introduce the deployment server, this is where no data reaches Splunk cloud and issuing cli commands the forwarding client just hangs. 

When I look in /etc/deployment-app/<app-folder>/default - there's no outputs.conf file on the deployment server and so I feel that the server config is missing something. 

I've used this guide as a setup reference for the deployment server but I still feel like I've missed something. 

https://docs.splunk.com/Documentation/SplunkCloud/8.1.2011/Admin/WindowsGDI (specifically step 3)

Any suggestions would be really appreciated,

Thanks! 

[achauhan2098]

Thanks for your help - I got this working.  I think the issue was ultimately with the forwarding.  It looks like the app for logging into Splunk Cloud was being sent to the UF. But I only got this working once I stripped the desired app down to just the inputs and outputs.conf files. 

I still have work to do but the base is there now. 

Thanks,

[achauhan2098]

Hi Giuseppe,

thanks for your response.

So do you mean I *need* to monitor the DS? and so does the outputs.conf file only need to be there if the DS itself is being monitored?  Because it's not in the chain of process to Splunk Cloud here is it? 

Thanks, Anish

[gcusello]

Hi @achauhan2098,

it's usually a best practice to monitor all your Splunk infrastructure, so I hint to redirect your DS's internal logs to Cloud, anyway you don't consume license!

If instead you don't want to monitor the DS, you don't need to have an outputs.conf in your DS.

It's different if you, (as hinted by best practices) deploy outputs.conf to your UFs using the DS: in this case you have to create a dedicated app (called e.g. TA_Forwarders or as you like) containing two files: outputs.conf and deploymentclient.conf.

In this case you have outputs.conf in $SPLUNK_HOME/etc/deployment-apps, but not in this folder, but in a TA (with the normal structure of a TA).

Ciao.

Giuseppe

[gcusello · Accepted solution]

Hi @achauhan2098 ,

the Deployment Server has the role to manage clients (Forwarders), but to monitor it, you have to consider it as a normal target and redirect its logs (through Intermediate Forwarders) to Indexers (on premise or Cloud), so outpus.conf should be in $SPLUNK_HOME/etc/system/local or (better) in a dedicated app.

The folder $SPLUNK_HOME/etc/deployment-app, it's used to contain the apps to deploy to the other Forwarders.

Infos at https://docs.splunk.com/Documentation/Splunk/8.1.1/Updating/Aboutdeploymentserver

Ciao.

Giuseppe

[gcusello]

Hi @achauhan2098,

good for you.

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉