Getting Data In

Windows Server 2008R2 Splunk server not receiving Windows Event Logs from a Windows 7 PC with Universal Forwarder installed

seeia
Engager

I initially tested the Splunk Server on a Windows 7 machine and installed the Universal Forwarder on another WIndows 7 machine.
This worked with no issues other than having to run sfc /scannow to get the Forwarder installed.

Now, I want to set up a permanent server on a Windows Server 2008R2 machine and am having issues. I am setting up the forwarder on the same Windows 7 machine I set up forwarding on before.

As soon as I installed the forwarder, I see data appear under 'Search' on the server. However, the data never updates and stays stale at the time of install. I noticed an error on the PC about a registry leak and thought this was the cause. I fixed the corrupted user profile on the PC but still the issue occurred.

I then looked at the splunkd.conf logs and noticed several TcpOut error informing the connection was refused. So, I added

[splunktcp://9997] 
connection_host = none

to the inputs.conf.

The "connection refused" errors are resolved but I still don't see updated data on the server.
I then looked more here and saw that adding this into inputs.conf could help:

[script://$SPLUNK_HOME\bin\scripts\splunk-wmi.path]
disabled = 0

[WinEventLog://Application]
disabled = 0

[WinEventLog://Security]
disabled = 0

[WinEventLog://System]
disabled = 0

[WinEventLog://ForwardedEvents]
checkpointInterval = 5
current_only = 0
disabled = 0
start_from = oldest

[WinEventLog://Setup]
checkpointInterval = 5
current_only = 0
disabled = 0
start_from = oldest

and also tried adding "wineventlog" to the Indexes on the server.
Still, no good.

I can telnet to the server so the connection seems to be working.
TcpOutputProc is reporting this:

11-21-2016 08:42:58.109 -0500 INFO  TcpOutputProc - Initializing connection for non-ssl forwarding to splunkserver:9997
11-21-2016 08:42:58.125 -0500 INFO  TcpOutputProc - tcpout group default-autolb-group using Auto load balanced forwarding
11-21-2016 08:42:58.125 -0500 INFO  TcpOutputProc - Group default-autolb-group initialized with maxQueueSize=512000 in bytes.
11-21-2016 08:42:58.921 -0500 INFO  TcpOutputProc - Connected to idx=10.5.1.181:9997

I see these warnings but not sure if they are relevant:

11-21-2016 08:41:54.532 -0500 WARN  ProcessRunner - Process with pid 2264 did not exit within a given grace period after being signaled to exit. Will have to forcibly terminate.
11-21-2016 08:42:55.348 -0500 WARN  AuditTrailManager - Private key file does not exist but is defined in audit.conf - no local event signing will take place. You can create auditTrail keys if necessary by running splunk createssl audit-keys
11-21-2016 08:42:55.847 -0500 INFO  LMTracker - Setting feature=ResetWarnings state=DISABLED_DUE_TO_LICENSE (featureStatus=2)
11-21-2016 08:42:58.063 -0500 WARN  PubSubMgr - Can't parse deployment server address from conf: ""
11-21-2016 08:42:58.281 -0500 WARN  UserManagerPro - Can't find [distributedSearch] stanza in distsearch.conf, using default authtoken HTTP timeouts
11-21-2016 08:42:58.655 -0500 WARN  X509Verify - X509 certificate (O=SplunkUser,CN=SplunkServerDefaultCert) should not be used, as it is issued by Splunk's own default Certificate Authority (CA). This puts your Splunk instance at very high-risk of the MITM attack. Either commercial-CA-signed or self-CA-signed certificates must be used; see: <http://docs.splunk.com/Documentation/Splunk/latest/Security/Howtoself-signcertificates>

The one that stands out to me the most is 11-21-2016 08:42:58.063 -0500 WARN PubSubMgr - Can't parse deployment server address from conf: ""
but would addressing this fix the problem?

Is there something else that I have missed?

0 Karma
1 Solution

seeia
Engager

Nevermind, I see what I did wrong.

I had set up the server to search an index I created called "test" for these logs.
The forwarder was configured to use "wineventlog" (per the above).

So the data is being forwarded now, I just can't see it displayed on the main Search UI by default.
If I run a manual search, I can see the data in index "wineventlog".

Now, I'm just wondering how to get this to appear in the UI. I'll open another question if I don't see how to do it.

View solution in original post

0 Karma

seeia
Engager

Nevermind, I see what I did wrong.

I had set up the server to search an index I created called "test" for these logs.
The forwarder was configured to use "wineventlog" (per the above).

So the data is being forwarded now, I just can't see it displayed on the main Search UI by default.
If I run a manual search, I can see the data in index "wineventlog".

Now, I'm just wondering how to get this to appear in the UI. I'll open another question if I don't see how to do it.

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...