Splunk forwarder failed to send logs from amazon linux instance

[meet_vadaria]

Trying to send logs to Splunk server using forwarder installed on Amazon Linux instances. I am not seeing any data on Splunk server. On forwarder side, I am seeing interfaces.sh related error in /var/log/splunkd.log.

09-10-2019 16:04:52.161 +0000 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/xxx_splunk_ta_nix/bin/interfaces.sh" cat: /sys/class/net/eth0/duplex: Invalid argument
09-10-2019 16:04:52.171 +0000 ERROR ExecProcessor - message from 
"/opt/splunkforwarder/etc/apps/xxx_splunk_ta_nix/bin/interfaces.sh" cat: /sys/class/net/eth0/speed: Invalid argument
09-10-2019 16:04:52.280 +0000 INFO  TailReader -   ...continuing.
09-10-2019 16:05:03.723 +0000 INFO  HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_xxxxxx_3D3B3E31-6E53-4D7A-AB7E-0FAF1FC62062
09-10-2019 16:05:10.007 +0000 WARN  FileClassifierManager - The file '/var/log/btmp' is invalid. Reason: binary.
09-10-2019 16:05:10.007 +0000 INFO  TailReader - Ignoring file '/var/log/btmp' due to: binary
09-10-2019 16:05:16.196 +0000 WARN  TailReader - Could not send data to output queue (parsingQueue), retrying...
09-10-2019 16:05:22.340 +0000 INFO  TailReader -   ...continuing.

I'm experiencing this issue with all amazon Linux servers.

[mguhad]

Hi,
If data is being terminated before it even reaches the parsing queue indicates a connection problem as data is being dropped before it enters the first pipeline. Please check your firewalls, ports, IP TABLES etc to root out connectivity issues between the indexers and UFs/HFs.
It also worth checking the security groups of the Indexers and the UFs if they are the same or have similar permissions/rules.

Hope this helps.

[meet_vadaria]

I don't think it's a port issue. as it's able to connect to indexer on 9997 port as per logs. I also verified with telnet.

09-10-2019 20:04:21.807 +0000 INFO TcpOutputProc - Connected to idx=100.117.33.54:9997, pset=0, reuse=0. using ACK.
09-10-2019 20:04:24.586 +0000 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/wss_splunk_ta_nix/bin/interfaces.sh" cat: /sys/class/net/eth0/duplex: Invalid argument
09-10-2019 20:04:24.603 +0000 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/wss_splunk_ta_nix/bin/interfaces.sh" cat: /sys/class/net/eth0/speed: Invalid argument
09-10-2019 20:05:18.036 +0000 INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_100.117.8.197_8089_100.117.8.197_mongop0-i-09e03c274a86ef49b-p1-ugw1.wss.symfedcloud.com_8323AF5D-B129-41AB-8B7B-8A9E95A9C7D0
09-10-2019 20:05:24.418 +0000 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/wss_splunk_ta_nix/bin/interfaces.sh" cat: /sys/class/net/eth0/duplex: Invalid argument
09-10-2019 20:05:24.419 +0000 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/wss_splunk_ta_nix/bin/interfaces.sh" cat: /sys/class/net/eth0/speed: Invalid argument
09-10-2019 20:05:31.624 +0000 INFO TcpOutputProc - Closing stream for idx=100.117.33.54:9997