Getting Data In

second instance of a heavy forwarder on the same system (UFW not able to connect)

Elsurion
Communicator

Hi all

I have a functional heavy forwarder on a systems, now i want a second heavy forwarder on the same system. I'd like to test some limiting features in the actual data stream. I cannot move it to another place, since i need some data throughput, which i don't have in any other non productive environment. And before i setup a new system, i like to take and utilize some installed hardware.

The setup of the second heavyforwarder worked well, including the binding of the second inputport as well the other second ports.
The port is open but the Universal Forwarder isn't able to open a stable connection to the second heavy forwarder.

This is the outputs.conf

splunk@mysystem:default $ cat outputs.conf
[tcpout]
# do not index locally
indexAndForward = false
# forward all loca indexes
forwardedindex.filter.disable = true
useACK = true
defaultGroup = splunk

[tcpout:splunk]
server = splunk-indexer01:9997

[tcpout:splunk-hfw]
server = splunk-hfw:9997

[tcpout:splunk-sink]
server = splunk-hfw:9996

The connection tests worked as well.

splunk@mysystem:default $  telnet splunk-hfw 9996
Trying <someip>...
Connected to splunk-hfw.
Escape character is '^]'.
^C
splunk@mysystem:default $

In the spunkd.log on either system i don't see anything which hitting me to the problem i have here...
On the UFW i'm getting time-outs

02-19-2018 13:42:00.425 +0100 WARN  TcpOutputProc - Cooked connection to ip=<hfwip>:9996 timed out
02-19-2018 13:42:00.425 +0100 WARN  TcpOutputProc - Cooked connection to ip=<hfwip>:9996 timed out

On the HFW i'm getting broken links, but i don't know why.

02-19-2018 13:42:16.849 +0100 ERROR TcpInputProc - Error encountered for connection from src=<ufw>:51891. Broken pipe

Traffic on the normal :9997 Port works without any issue
Anyone has any hint for the problem?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas     Cisco Live 2026 is almost here, and this ...

What Is the Name of the USB Key Inserted by Bob Smith? (BOTS Hint, Not the Answer)

Hello Splunkers,   So you searched, “what is the name of the usb key inserted by bob smith?”  Not gonna lie… ...

Automating Threat Operations and Threat Hunting with Recorded Future

    Automating Threat Operations and Threat Hunting with Recorded Future June 29, 2026 | Register   Is your ...