Getting Data In

Why are my Windows system event logs not getting indexed and getting "WARN TcpOutputProc - Raw connection to ip=serverip:9997 timed out"?

SridharS
Path Finder

Hi, I need to index some Windows system event logs of a remote server (using forwarder) into Splunk. My files are as below in the path $home/etc/apps/<myappname>/local on the forwarder

inputs.conf

[winEventLog://system]
disable = 0
whitelist = 41,104
index = wineventlog

outputs.conf

[tcpout]
default = primary_indexers
[tcpout:primary_indexers]
useACK=true
server = splunkservername:9997

I restarted the forwarder. My tcp port 9997 is open, but why are the logs not getting indexed in Splunk. The last log in the splunkd was:

12-15-2015 14:45:37.097 -0500 WARN  TcpOutputProc - Raw connection to ip=serverip:9997 timed out
12-15-2015 14:45:37.097 -0500 INFO  TcpOutputProc - Ping connection to idx=serverip:9997 timed out. continuing connections
12-15-2015 14:45:37.097 -0500 INFO  TcpOutputProc - Ping connection to idx=serverip:9997 timed out. continuing connections
0 Karma
1 Solution

hortonew
Builder

Has it ever been able to send logs to your indexers? If not, verify that your universal fowarder can:

telnet splunkservername 9997

If that is successful, rule out slow/problematic dns resolutions by configuring your hosts file to map that server name to its IP. Report back after ruling these out. If you try both of these and there are still issues, chances are the problem is on the indexer and not your client sending data. You can also try setting connection_host=false in your inputs.conf for the 9997 stanza, making sure the indexer isn't trying to resolve the name of the forwarder.

View solution in original post

hortonew
Builder

Has it ever been able to send logs to your indexers? If not, verify that your universal fowarder can:

telnet splunkservername 9997

If that is successful, rule out slow/problematic dns resolutions by configuring your hosts file to map that server name to its IP. Report back after ruling these out. If you try both of these and there are still issues, chances are the problem is on the indexer and not your client sending data. You can also try setting connection_host=false in your inputs.conf for the 9997 stanza, making sure the indexer isn't trying to resolve the name of the forwarder.

SridharS
Path Finder

it worked when i added connection_host=false to my inputs.conf

0 Karma

goelt2000
Explorer

hi,

Whats the difference between connection_host=false and connection_host=ip.

I am facing similar errors as mentioned on
https://answers.splunk.com/answers/49833/splunk-forwarder-connection-refused-from-splunk-indexer.htm...

Thanks

0 Karma

hortonew
Builder

From the docs:
connection_host = [ip|dns|none]
* "ip" sets the host to the IP address of the system sending the data.
* "dns" sets the host to the reverse DNS entry for IP address of the system
sending the data.
* "none" leaves the host as specified in inputs.conf, typically the splunk
system hostname.
* Defaults to "dns".

0 Karma

goelt2000
Explorer

none means it will leave it to be the receiver hostname?
What does connection_host=false, means?

Thanks much!

0 Karma

hortonew
Builder

None would be whatever is configured in the forwarder's inputs.conf. So if you have a universal forwarder sending data, whatever is in that universal forwarder's inputs.conf for hostname.

I'm 90% sure none and false do the same thing, but I couldn't find confirmation.

0 Karma
Get Updates on the Splunk Community!

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...

Explore the Latest Educational Offerings from Splunk (November Releases)

At Splunk Education, we are committed to providing a robust learning experience for all users, regardless of ...

New This Month in Splunk Observability Cloud - Metrics Usage Analytics, Enhanced K8s ...

The latest enhancements across the Splunk Observability portfolio deliver greater flexibility, better data and ...