Deployment Architecture

The Client forwarder management not showing the clients

AAlhabba
Explorer

Dears,

       After upgraded Splunk from 9.1.2 version to 9.2.0 version, the deployment server not showing the clients, but Splunk receiving logs from clients, and also the client agents showing on all Splunk servers under setting --> Forwarder Managment except Deployment server, I don't know how that occurred, I didn't change anything.

Kindly your support for that.

 

Best Regards, 

Labels (1)
1 Solution

AAlhabba
Explorer

Dears,

 

        I have resolved the issue by adding below configuration under outputs.conf in the deployment server, then restart splunk service in the deployment server.

 

[indexAndForward]

index = true

selectiveIndexing = true

 

 

You can see below URL:

 

Upgrade pre-9.2 deployment servers - Splunk Documentation

View solution in original post

askargbo
Engager

I have tried all the recommendations in this thread and non of them works. I upgraded from 9.0 to 9.3, but the clients are not phoning in.

0 Karma

JoshuaJJ
Path Finder

This is not working for me 😞 

I have the app in place on my DS (/etc/apps/DS_Fix/local/outputs.conf 

0 Karma

kell_cena
Explorer

@AAlhabba , thank you for the solution .Worked like a charm.

AJ_splunk1
New Member

Does this work with splunk cloud as well? WE have splunk onprem deployment server, indexers are all in the cloud and experiencing the same where clients are not showing up after an update to 9.2.x. They are phoning home however as per the logs

0 Karma

AJ_splunk1
New Member

nvm figured it out. It was the output.conf in this app - etc/apps/SplunkDeploymentServerConfig. Documentation is a bit confusing for this

0 Karma

learningmode
Loves-to-Learn Everything

Hi @AJ_splunk1, just a heads-up that https://docs.splunk.com/Documentation/Splunk/9.2.0/Updating/Upgradepre-9.2deploymentservers states that " In particular, there is a new system-generated app, etc/apps/SplunkDeploymentServerConfig, which contains configuration files necessary to the proper functioning of the deployment server. Do not alter this directory or its files in any way."

I chose to implement the 9.2 fix (also shown on the page link above) as a separate app in \etc\apps on the ds and just called something like "Fix_DSClientList" so it's more obvious there's a modification in place.

I hope that helps.

0 Karma

AAlhabba
Explorer

Dears,

 

        I have resolved the issue by adding below configuration under outputs.conf in the deployment server, then restart splunk service in the deployment server.

 

[indexAndForward]

index = true

selectiveIndexing = true

 

 

You can see below URL:

 

Upgrade pre-9.2 deployment servers - Splunk Documentation

andrewtrobec
Motivator

Boy am I glad to have found this thread.  Got my problem solved, thank you so much ❤️

JRW
Splunk Employee
Splunk Employee

Another fix to try is as follows:

Find your distsearch.conf and then find the stanza that has default = true in it. In that stanza, make sure localhost:localhost is listed in the setting below

 

 

servers = 

 

 

 

For example, it was like this before:

distributedSearch:testgroup1
default = true
servers = somehostname.company.com

Once you find that stanza, add localhost to make it look like this (and it's literal in that it's simply localhost:localhost)

distributedSearch:testgroup1
default = true
servers = somehostname.company.com, localhost:localhost

Restart the DS and from the internal thread within a few minutes/hours the clients should start to populate again

 

JoshuaJJ
Path Finder

This worked instantly!  I appreciate you! 

Thanks, 

JJJ 

0 Karma

jmcnee
Engager

This was so helpful and fixed my problem, thankyou very much!

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Thereare no miracles. If they are showing in the Forwarder Management section on a server different than your designated DS, they must have been pointed there somehow. Check your deployment server definition on your forwarders.

0 Karma

landrujw
Explorer

We are experiencing the same thing. The clients are showing up in the client_events logs checking in and phoning home on the deployment server. But after updating to 9.2 they aren't appearing under the Settings>Forwarder Management page on the DS. We have not made any changes to the forwarders yet.

0 Karma

mykol_j
Communicator

FWIW, happening here as well, with 9.2.0.1.

Checked all The Things mentioned in that doc everyone keeps referencing, including those stanzas mentioned numerous times here.

Another symptom of mine is that the ForwarderManager (deployer) doesn't appear in my monitored servers in the SplunkManager (aka Master).

Tags (1)
0 Karma

mykol_j
Communicator

This is nuts. Go figure.

I ended up fixing this my removing the "Deployment Server" role from the system, saving it, then adding it back, restarting the service, bam! Fixed.

I'd rather be lucky than good...

JoshuaJJ
Path Finder

Hmmm, are you talking about the role defined within the Monitoring Console? I am having tons of issues resolving this. 

0 Karma

AAlhabba
Explorer

Hi,

Did you open case with Splunk support about this issue, I already opened still Splunk support trying to resolve it.

 

Best Regards,

0 Karma

ccsfdave
Builder

Any luck with support?  I tried the outputs.conf solution in this thread but it doesn't seem to have worked.  

 

Pre-upgrade from 9.0.x to 9.2.1 I had 300ish clients in my DS.  right now only 14 are showing up.

 

Thanks,

Dave

0 Karma

landrujw
Explorer

Applying the stanza you referenced below worked for us as well:
 

[indexAndForward]
index = true
selectiveIndexing = true   

 Thanks!

computermathguy
Path Finder

Created a local directory within the SplunkDeploymentServerConfig app.  Added the outputs.conf

/opt/splunk/etc/apps/SplunkDeploymentServerConfig/local/outputs.conf

[indexAndForward]
index = true
selectiveIndexing = true



Clients started reporting to the DS after restarting Splunk.  Thankful I found this thread.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...