Getting Data In

How to troubleshoot why a Windows universal forwarder can forward permon data to Splunk Light 6.3, but not Windows event logs?

acrismatic
New Member

Hi to all,

I'm a newbie with Splunk this week, and trying to configure a forwarder in W2008 in order to forward event logs to Splunk Light 6.3 configured as an indexer in Centos.

I've installed the universal forwarder and set the server in outputs.conf:

tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = 192.168.1.175:9997

[tcpout-server://192.168.1.175:9997]

That's my inputs.conf in Splunk Server :

[splunktcp://192.168.1.102:9997]
connection_host = ip

# Windows platform specific input processor.

[WinEventLog://Application]
disabled = 0
#current_only = 1
index = wineventlog

[WinEventLog://Security]
disabled = 0
#current_only = 1
index = wineventlog

[WinEventLog://System]
disabled = 0
#current_only = 1
index = wineventlog

Well, the case is I can't seem to make my w2008 send event logs to the indexer, but perfmon events appear on it. I've set this info during universal forwarder installation, setting the ip and port for indexer, and setting Windows events logs and perfmon.

In Splunk Web, I'm trying to configure data input > forwarders > Windows events logs, but it says there's no forwarder available.

On searches, I can see perfmon events, but not so much...

Some tries I've done:

  • I've use netstat to test if the connection is ok, and it is in ESTABLISHED state (so i'm receiving perfmon successfully).
  • Check indexes. It's strange for me the result. I can see how wineventlog index is growing constantly, but no event appears in the search page. All perfmon events are sent to the main index.
  • Review splunkd.log in w2008 forwarder and in Centos Splunk Server. I've found some errors about connections, but no idea how to solve it. This error appears in the forwarder:

    10-16-2015 11:28:37.342 +0200 INFO TcpOutputProc - Connection to 192.168.1.175:9997 closed. Connection closed by server.
    10-16-2015 11:28:38.853 +0200 WARN TcpOutputFd - Connect to 192.168.1.175:9997 failed. No connection could be made because the target machine actively refused it.
    10-16-2015 11:28:38.853 +0200 ERROR TcpOutputFd - Connection to host=192.168.1.175:9997 failed
    10-16-2015 11:28:38.853 +0200 WARN TcpOutputProc - Applying quarantine to ip=192.168.1.175 port=9997 _numberOfFailures=2
    10-16-2015 11:28:49.855 +0200 INFO TcpOutputProc - Removing quarantine from idx=192.168.1.175:9997
    10-16-2015 11:28:50.357 +0200 INFO TcpOutputProc - Connected to idx=192.168.1.175:9997
    And this appears in the indexer:

    10-16-2015 11:28:21.791 +0200 INFO TcpInputProc - Waiting for connection from src=192.168.1.102:57931 to close before shutting down TcpInputProcessor.
    10-16-2015 11:28:23.286 +0200 ERROR TcpInputProc - Error encountered for connection from src=192.168.1.102:57931. Local side shutting down
    10-16-2015 11:28:35.883 +0200 INFO TcpInputConfig - performing DNS lookup on 192.168.1.102
    I'm a bit confused with these errors. How can I receive perfmon events if indexer is refusing connections from forwarder?

Two servers are in the same subnet, with booth firewalls deactivated. Now I'm at a point where I don't know what else to check. Could someone give me some advice to look for?

I've tried to give all information possible. Don't hesitate to ask for more information, remember I'm newbie with Splunk and I'm sure I'm loosing configs and things to do...

Best Regards,

0 Karma

jterry
Splunk Employee
Splunk Employee

what does your search look like?
Are you specifying the wineventlog index? (index=wineventlog *)

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Event Series: Telemetry Pipeline Management

Balancing Scale and Spend: Gaining Control Over High-Volume Metrics in Splunk Observability Cloud As ...

Kick the Tires Before You Commit: A Hands-On Tour of the Splunk Observability Cloud ...

Evaluating an enterprise observability platform usually goes like this: fill out a form, get a free trial with ...

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...