I've encountered similar issue with slightly different logs as follows:
11-26-2013 23:03:58.619 +0000 WARN TcpOutputProc - Shutdown timed out for 107.22.10.147:9997
11-26-2013 23:04:28.607 +0000 WARN TcpOutputProc - Raw connection to ip=107.22.10.147:9997 timed out
11-26-2013 23:04:28.607 +0000 INFO TcpOutputProc - Detected connection to 107.22.10.147:9997 closed
11-26-2013 23:04:28.607 +0000 INFO TcpOutputProc - Will close stream to current indexer 107.22.10.147:9997
11-26-2013 23:04:28.607 +0000 INFO TcpOutputProc - Closing stream for idx=107.22.10.147:9997
11-26-2013 23:04:58.929 +0000 WARN TcpOutputProc - Cooked connection to ip=107.20.29.58:9997 timed out
11-26-2013 23:05:18.931 +0000 WARN TcpOutputProc - Cooked connection to ip=23.23.14.246:9997 timed out
11-26-2013 23:05:38.931 +0000 WARN TcpOutputProc - Cooked connection to ip=107.22.148.176:9997 timed out
11-26-2013 23:05:58.933 +0000 WARN TcpOutputProc - Cooked connection to ip=54.243.3.115:9997 timed out
11-26-2013 23:06:18.936 +0000 WARN TcpOutputProc - Cooked connection to ip=50.17.56.245:9997 timed out
11-26-2013 23:06:20.068 +0000 INFO TcpOutputProc - Connected to idx=54.224.46.188:9997 using ACK.
11-26-2013 23:06:58.937 +0000 WARN TcpOutputProc - Cooked connection to ip=23.22.208.232:9997 timed out
11-26-2013 23:07:18.940 +0000 WARN TcpOutputProc - Cooked connection to ip=107.20.29.58:9997 timed out
11-26-2013 23:07:20.942 +0000 INFO TcpOutputProc - Connected to idx=23.20.94.208:9997 using ACK.
11-26-2013 23:07:28.600 +0000 WARN TcpOutputProc - Shutdown timed out for 23.23.14.246:9997
11-26-2013 23:07:40.590 +0000 INFO TcpOutputProc - Connected to idx=54.224.135.13:9997 using ACK.
If I'm using splunk>storm then I won't be able to see indexers splunkd.log and metrics.log to see what is going on with the indexer, right? If so, where do I go from here?
My setup is as follows:
webserver nodes->splunkforwarder->splunkstorm
Both webserver nodes and splunkforwarder shows active forwards.
Any help is greatly appreciated.
Thank you.
... View more