topic Re: Splunk Cloud Trial: Why am I getting "ERROR TcpOutputFd - Connection to host=(splunk-cloud-ip):9997 failed" after setting up a universal forwarder on our EC2 instance? in Getting Data In

Splunk Cloud Trial: Why am I getting "ERROR TcpOutputFd - Connection to host=(splunk-cloud-ip):9997 failed" after setting up a universal forwarder on our EC2 instance?

pjoiner — Tue, 17 Nov 2015 04:10:28 GMT

I signed up for a Splunk Cloud trial, and set up a universal forwarder on one of our EC2 instances. However, I keep getting this in splunkd.log:

ERROR TcpOutputFd - Connection to host=[ip address of input server]:9997 failed

I tried telnet to the ip/port and it was successful, so there should be no network-related issues.

If I go in the admin console to Settings->Forwarding and Receiving I see the message:

There was an error retrieving the configuration, can not process this page

Is there some additional configuration on either the admin or on our EC2 instance (universal forwarder) to get this to work? Or does the Splunk Cloud trial not allow contributing data to the instance?

Re: Splunk Cloud Trial: Why am I getting "ERROR TcpOutputFd - Connection to host=(splunk-cloud-ip):9997 failed" after setting up a universal forwarder on our EC2 instance?

pjoiner — Wed, 18 Nov 2015 01:13:39 GMT

BTW the universal forwarder is running on Amazon Linux with the latest OS updates.

Re: Splunk Cloud Trial: Why am I getting "ERROR TcpOutputFd - Connection to host=(splunk-cloud-ip):9997 failed" after setting up a universal forwarder on our EC2 instance?

Shinpo — Wed, 18 Nov 2015 08:56:29 GMT

I have the same problem ... anyone can help us ?

Best
Giovanni

Re: Splunk Cloud Trial: Why am I getting "ERROR TcpOutputFd - Connection to host=(splunk-cloud-ip):9997 failed" after setting up a universal forwarder on our EC2 instance?

Sebastian2 — Wed, 18 Nov 2015 15:01:13 GMT

Could you give us a little more information? outputs.conf of the forwarder and inputs.conf of the receiver? and maybe the $SPLUNK_HOME/etc/apps/search/metadata/local.meta

Re: Splunk Cloud Trial: Why am I getting "ERROR TcpOutputFd - Connection to host=(splunk-cloud-ip):9997 failed" after setting up a universal forwarder on our EC2 instance?

Shinpo — Wed, 18 Nov 2015 18:07:51 GMT

outputs.conf
/opt/splunkforwarder/etc/system/local/outputs.conf

[tcpout]
defaultGroup = splunkcloud
disabled = false
maxQueueSize = 1500
indexAndForward = false
[tcpout:splunkcloud]
server = input-prd-p-pdsmk7bx6vlg.cloud.splunk.com:9997

inputs.conf
/opt/splunkforwarder/etc/system/local/inputs.conf

[default]
host = SyslogSRV
[monitor:///var/log/TEST-SYSLOG/test-sysLog.log]
[splunktcp-ssl://9997]

local.meta

[inputs/monitor%3A%2F%2F%2Fvar%2Flog%2FTEST-SYSLOG%2Ftest-sysLog.log]
owner = splunk-system-user
version = 6.3.1
modtime = 1447503002.498094000

[inputs/monitor%3A%2F%2F%2Fvar%2Flog%2Fsyslog]
owner = admin
version = 6.3.1
modtime = 1447516194.527718000

[inputs/monitor%3A%2F%2F%2Fvar%2Flog]
owner = admin
version = 6.3.1
modtime = 1447767752.634803000

[inputs/splunktcp%3A%2F%2F9997]
owner = admin
version = 6.3.1
modtime = 1447857226.751613000

Re: Splunk Cloud Trial: Why am I getting "ERROR TcpOutputFd - Connection to host=(splunk-cloud-ip):9997 failed" after setting up a universal forwarder on our EC2 instance?

pjoiner — Thu, 19 Nov 2015 02:35:37 GMT

I found the cause of my issue. Although I could telnet to port 9997, the problem was that the EC2 instance did not have a direct path to the internet (i.e. was using a proxy). To test I used a different EC2 that had a direct path to the internet, and the forwarder started working correctly.

I had been told that there was no internet proxy/firewall for the first machine I had tried, but that information was not correct.