TCP Data Input is not accepting connections

[sephora_it]

I have setup port 9990 as a TCP Data Input, but our Splunk server is not accepting connections from that port. It works from localhost on the server itself, but not from any other machine even on the same subnet. There is no firewall dropping the traffic. Any idea what may be wrong?

Works from localhost:

[root@tdcvlog01 ~]# telnet 127.0.0.1 9990
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

Not from external hosts:

C:\Users\splunk>telnet x.x.x.x 9990
Connecting To x.x.x.x...Could not open connection to the host, on port 9990: Connect failed

[root@tdcvlog01 ~]# netstat -ntap

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name

tcp 0 0 0.0.0.0:9997 0.0.0.0:* LISTEN 1087/splunkd

tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1205/rpcbind

tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1479/sshd

tcp 0 0 0.0.0.0:52502 0.0.0.0:* LISTEN 1223/rpc.statd

tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 1359/cupsd

tcp 0 0 0.0.0.0:8089 0.0.0.0:* LISTEN 1087/splunkd

tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1563/master

tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 1167/python

tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN 1087/splunkd

tcp 0 0 0.0.0.0:9990 0.0.0.0:* LISTEN 1087/splunkd

tcp 0 0 0.0.0.0:5672 0.0.0.0:* LISTEN 1606/qpidd

[sowings · Accepted solution]

Sounds like a firewall issue. Typically the loopback interface (127.0.0.1) is permitted to send / receive connections from the network, but access to the other interfaces may not be permitted. Check your firewall settings (iptables, ipchains, etc).

[sephora_it]

Disabling iptables did the trick. Good call and thanks so much for your help!

[sephora_it]

There is no forwarder. We are sending log files from an iPhone directly to our Splunk indexer on port 9990.

Right now our splunkd.log is being flooded by some error messages, but I do not think they are related. I cannot find any logs related to my issue.

08-13-2013 16:41:32.757 -0700 ERROR SearchParser - Could not find macro 'conf-change' that takes 0 arguments. Expecting stanza name 'conf-change'.

08-13-2013 16:42:04.390 -0700 WARN SearchOperator:kv - Invalid key-value parser, ignoring it, transform_name='rbsh_process'

[lukejadamec]

What are you seeing in the splunkd.log files on the forwarder and indexer?