External Forwarder sending to DMZ Heavy Forwarder (9997)

[jordanking1992]

All,

I am trying to send to from an external forwarder to a DMZ Heavy Forwarder that is behind a firewall w/ 9997 open. In my attempt to send data, I am getting "TCP output processor has paused the data flow" and unable to get  to get results to the DMZ forwarder.

I have similar setup with HEC and Syslog that works just fine but for traffic on 9997, Splunk is refusing to send. The box is ping-able and 9997 is open. The firewall is set up to route to the DMZ host as well.

Anyone have some  ideas on what I may be doing wrong or things to consider when setting up a Splunk HF in a DMZ?

[thambisetty]

can you verify splunk listen is enabled on Splunk DMZ Heavy forwarder. by running below command 

$SPLUNK_HOME/bin/splunk cmd btool inputs list | grep 9997

output should be like below:

[splunktcp://9997]

if you don't see the above output, that means 9997 is not enabled on DMZ HF, but port is opened on firewall.

you can enable by running below command repalce username and password with actual.

splunk enable listen 9997 -auth <username>:<password>

Try to generate some traffic on 9997 to DMZ HF from where you are saying telnet is working.

telnet yourdmzip 9997 -> enter

check if it says connected or not, if its connected, type something random text and don't close the session.

check what IP you are seeing in DMZ HF by doing netstat command 

netstat -anp | grep 9997