- Splunk Answers
- :
- Splunk Administration
- :
- Deployment Architecture
- :
- Re: Number of Forwarders not Increasing?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Number of Forwarders not Increasing?
Hey All,
I have sent out a deployment of the forwarder to a couple hundred machines via GPO. All seemed to be going OK except that since the number of forwarders has reached 108 no further forwarders are appearing. The installs are the same and are completing successfully. Is there a limit on the number of forwarders?
Regards,
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Have a read of, http://splunk-base.splunk.com/answers/4097/is-there-a-maximum-number-of-forwarders-per-indexer
Hopefully that should answer your questions 🙂
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well, like Ayme says above you need to have a look at the deployment server end to see what it is saying but if the connections are being rejected the chances are its probably a TCP socket issue
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Unfortunately not, i have 108 clients forwarding successfully to the server, just application and system logs, which is about 4mb a day a day at the moment. Should be plenty of room for more clients from them numbers. Just not sure why the server is rejecting the additional ones.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There is no limit from Splunk as such (although there may be limits at the system level based on number of connections to Indexer).
You really want to check the $SPLUNK_HOME/var/log/splunk/splunkd.log on one of the non-communicating Forwarders and see where it complains of not being able to connect.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Strangely, even the ones i can see in the deployment manager as active report that the server refused the connection?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have compared the contents of input.conf in the following location
c:\Program Files\SplunkUniversalForwarder\etc\system\local\inputs.conf
However they both seem the same with the right host. Unless i am looking at the wrong inputs.conf?
A working computer.
Contains
[default]
host = IT1-05-0847
A missing machine
[default]
host = Admiss-02-0674
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks, the search index=_internal | stats count by host
Does indeed display the missing hosts and the errors. For example
03-23-2012 10:11:43.501 +0000 WARN PubSubConnection - Cannot convert str: to a valid status, returning eRejected.
host=6F01-01-1560 Options| sourcetype=splunkd
Options| source=C:\Program Files\SplunkUniversalForwarder\var\log\splunk\splunkd.log
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well, certainly looks to be connected. Chances are it just hasn't sent any data. The search:
index=_internal | stats count by host
...should show you that it is reporting internal messages, in which case you just need to ensure you have well configured monitors in the inputs.conf of the "missing" forwarders (could the fact that it is not connecting to the Deployment Server be the reason for this?)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
16.103 is the correct server and 9997 is the correct port..
03-23-2012 09:53:06.363 +0000 INFO TcpOutputProc - Connected to idx=192.168.16.103:9997
03-23-2012 09:53:11.285 +0000 WARN DeploymentClient - Unable to send handshake message to deployment server. Error status is: rejected
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The above messages relate to the Deployment Server so you don't want to look at the messages for the "DeploymentClient" (for the time being). Restart the Forwarder and look at the log file again - you should see messages related to "TcpOut".
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Anyone able to suggest why the server would rejecting additional forwarders?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ive Checked the outputs.conf on the client and that is pointing to the correct server and port. the splunkd.log says ...
03-22-2012 10:20:14.085 +0000 WARN PubSubConnection - Cannot convert str: to a valid status, returning eRejected.
03-22-2012 10:20:25.804 +0000 WARN DeploymentClient - Unable to send handshake message to deployment server. Error status is: rejected
03-22-2012 10:20:37.819 +0000 WARN DeploymentClient - Unable to send handshake message to deployment server. Error status is: rejected
03-22-2012 10:20:49.819 +0000 WARN DeploymentClient - Unable to send handshake message to deployment server. Error status is: rejected
03-22-2012 10:21:01.819 +0000 WARN DeploymentClient - Unable to send handshake message to deployment server. Error status is: rejected
Welcome to the Splunk Community!
Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM
Adoption of RUM and APM at Splunk
