forwarder configuration with Deployment app issue

[k_harini]

Hi,
We have configured forwarder with deployment app.. It was working for _internal index. when I add new inputs to the app its not getting indexed..

splunkd log error -
WARN HttpListener - Socket error from 172.17.142.83 while accessing /en-GB/splunkd/__raw/servicesNS/harini.k/Application_Monitoring/search/parser: Connection closed by peer

Deployment client, Deployment server, server class all set.. what would be the issue.. please help

[inventsekar]

from another post(link below) of the similar issue -

As it turns out, Splunk 6.0's reworked REST HTTP server introduces new self-imposed limits on the number of threads and sockets it allows itself to use. This is visible on startup in splunkd.log:

 INFO  loader - Limiting REST HTTP server to 341 sockets
 INFO  loader - Limiting REST HTTP server to 341 threads

These are roughly set to one third of the open file descriptor limit imposed on splunkd by the operating system. Here, we had an open file descriptor limit of 1,024, which resulted in a self-imposed limit of 341 threads and sockets:

 INFO  ulimit - Limit: open files: 1024 files

When one of these limits is hit, splunkd won't be able to honor further REST API calls and many things can go wrong. To prevent this from happening, one should raise or lift the per-process file descriptor limit on systems that are dedicated to running Splunk. Alternatively, one can also change the way these self-imposed limits are put in place by splunkd in server.conf:

 maxThreads = 
     * Number of threads that can be used by active HTTP transactions.
       This can be limited to constrain resource usage.
     * If set to 0 (the default) a limit will be automatically picked
       based on estimated server capacity.
     * If set to a negative number, no limit will be enforced.
 maxSockets = 
     * Number of simultaneous HTTP connections that we'll accept simultaneously.
       This can be limited to constrain resource usage.
     * If set to 0 (the default) a limit will be automatically picked
       based on estimated server capacity.
     * If set to a negative number, no limit will be enforced.

https://answers.splunk.com/answers/105292/what-is-the-cause-of-these-socket-errors-reported-in-splun...

[k_harini]

In forwarder splunkd log no error - I could see plenty of "06-03-2017 17:02:46.516 +1000 INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_172.28.152.187_8089_172.28.152.187_WUWFMWIM002-C_9BFE906E-2849-419A-B269-4F7D5223A0A9" message..

[inventsekar]

///// In forwarder splunkd log no error - I could see plenty of "06-03-2017 17:02:46.516 +1000 INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_172.28.152.187_8089_172.28.152.187_WUWFMWIM002-C_9BFE906E-2849-419A-B269-4F7D5223A0A9" message.. /////

these Running phone (and phonehome) messages are good. they mean the deployment client was communicating(just like a normal "phone") with the Deployment Server.

[dineshraj9]

Can you check if the input app has reached the forwarders $SPLUNK_HOME/etc/apps directory?

You can check the list of monitors on any forwarder by running this -

$SPLUNK_HOME/bin/splunk list monitor -auth admin:password

[k_harini]

I could see my input folder path here