Getting Data In

forwarder stopped sending to indexer but continues to send to 3rd party receiver

Log_wrangler
Builder

Any help on this is greatly appreciated.

I have a bunch of servers with UFs sending to a HF that is configured to send cooked data to splunk indexers and raw uncooked to a 3rd party receiver.

The flows have been running fine, but suddenly data to the indexers stopped.

I am looking for any suggestions to troubleshoot this.

Thank you

0 Karma
1 Solution

Log_wrangler
Builder

Thanks, did that and could see that the 3rd party receiving ip was blocking, so there must have been a build up that ultimately stopped the forwarder...

02-22-2018 16:44:56.119 -0600 INFO TcpOutputProc - Connected to idx=1.1.1.1:some_port
02-22-2018 16:53:11.041 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 10 seconds.
02-22-2018 16:53:21.024 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 20 seconds.
02-22-2018 16:54:09.009 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 40 seconds.

View solution in original post

0 Karma

Log_wrangler
Builder
  • should mention, that eventually everything stopped receiving, after a few minutes.
0 Karma

Log_wrangler
Builder

Thanks, did that and could see that the 3rd party receiving ip was blocking, so there must have been a build up that ultimately stopped the forwarder...

02-22-2018 16:44:56.119 -0600 INFO TcpOutputProc - Connected to idx=1.1.1.1:some_port
02-22-2018 16:53:11.041 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 10 seconds.
02-22-2018 16:53:21.024 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 20 seconds.
02-22-2018 16:54:09.009 -0600 WARN TcpOutputProc - Forwarding to indexer group Subsidiary blocked for 40 seconds.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Check the logs.

0 Karma
Get Updates on the Splunk Community!

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...

Splunk and Fraud

Watch Now!Watch an insightful webinar where we delve into the innovative approaches to solving fraud using the ...