Getting Data In

forwarder stopped sending to indexer but continues to send to 3rd party receiver

Builder

Any help on this is greatly appreciated.

I have a bunch of servers with UFs sending to a HF that is configured to send cooked data to splunk indexers and raw uncooked to a 3rd party receiver.

The flows have been running fine, but suddenly data to the indexers stopped.

I am looking for any suggestions to troubleshoot this.

Thank you

0 Karma
1 Solution

Builder

Thanks, did that and could see that the 3rd party receiving ip was blocking, so there must have been a build up that ultimately stopped the forwarder...

02-22-2018 16:44:56.119 -0600 INFO TcpOutputProc - Connected to idx=1.1.1.1:some_port
02-22-2018 16:53:11.041 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 10 seconds.
02-22-2018 16:53:21.024 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 20 seconds.
02-22-2018 16:54:09.009 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 40 seconds.

View solution in original post

0 Karma

Builder
  • should mention, that eventually everything stopped receiving, after a few minutes.
0 Karma

Builder

Thanks, did that and could see that the 3rd party receiving ip was blocking, so there must have been a build up that ultimately stopped the forwarder...

02-22-2018 16:44:56.119 -0600 INFO TcpOutputProc - Connected to idx=1.1.1.1:some_port
02-22-2018 16:53:11.041 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 10 seconds.
02-22-2018 16:53:21.024 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 20 seconds.
02-22-2018 16:54:09.009 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 40 seconds.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Check the logs.

0 Karma