Splunk Universal Forwarder - basic Windows install

[jamescrowley]

Hi everyone,

I've just manually installed our first Windows-based Splunk Universal Forwarder. I checked the boxes asking for various Windows event logs, and opted-in to the Windows extension it suggests.

However, I can't get it forwarding to splunk. The machine itself can connect on port 8089 to the deployment server specified. Looking in the logs, I see an entry with

07-06-2014 12:39:02.186 +0000 ERROR TcpOutputProc - LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf.
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - EvtDC::bind: Failed to get domain controller name with DsGetDcName: (1355)
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - EvtDC::bind: Failed to get domain controller name with DsGetDcName: (1355)
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - EvtDC::bind: Failed to get domain controller name with DsGetDcName: (1355)
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - EvtDC::connectToDC: DsBind failed: (1355)
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventLogChannel::init: Failed to bind to DC, dc_bind_time=0 msec
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - EvtDC::connectToDC: DsBind failed: (1355)
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - EvtDC::connectToDC: DsBind failed: (1355)
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventLogChannel::init: Failed to bind to DC, dc_bind_time=0 msec
07-06-2014 12:39:08.083 +0000 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventLogChannel::init: Failed to bind to DC, dc_bind_time=0 msec

However, my understanding was the default windows install should be configuring outputs.conf for me? Also, I'm not sure whether the DC binding errors matter (this machine isn't on a domain). Any idea what's going wrong?

Thanks

[dstaulcu]

I don't think the dc_bind would prevent receipt of events.

Are you sure your receivers are able to receive events? Are you receiving events from other host types? Have you enabled receiving? On same port specified by client?

Run ".\bin\splunk cmd btool outputs list" from the command line on your windows client. Are the correct server names:ports specified? Can you reach those server names:ports from client via ping and telnet?

[dstaulcu]

Yes. There should be configuration details in outputs.conf describing the server(s) to which events should be sent.

You can find the spec for outputs.conf here:
http://docs.splunk.com/Documentation/Splunk/6.1.1/Admin/Outputsconf

At the bottom of the outputs.conf spec file you will find examples showing the minimum info needed.

The splunk universal for windows has default inputs which are routed to the _internal index.

Once you get outputs functioning you can go to your search head and search index=_internal host="yourwindowshostname" to verify that events are searchable

[jamescrowley]

Sorry @dstaulcu if I'm missing something here - but specifying the deployment client (IP + port) is the only thing I have done during the install of the universal forwarder? I haven't touched anything else? That's why I'm struggling to understand what's going wrong here

[dstaulcu]

I don't recall where it should be by virtue of the specification via installer. What I do remember of use of specification of confs via installer is that the installer places the confs in a location which is difficult to manage (override) over time. Better to specify only deploymentclient details (use a DNS alias) via installer and to have the deploymentclient download desired deployment-apps (outputs, inputs) on first phoneHome.

Save yourself some trouble down the road and take this opportunity to push your desired inputs/outputs via deployment server instead of relying on installer to do so.

[jamescrowley]

All the settings being listed by btool appear to come from

etc/system/default/outputs.conf

There is no outputs.conf in etc/system/local. Should there be? And if so, any idea why the installer hasn't added it? Thanks!

[jamescrowley]

I just have a standard Splunk install running on a Linux AMI (basic install using the rpm package). The port is definitely accessible and accepting connections.

On the windows machine, I have

[target-broker:deploymentServer]
targetUri = XXXX:8089

set in /etc/system/local/deploymentclient.conf

I also ran btool outputs list (wasn't quite sure which command you wanted me to run), which just has a [tcpout] section (I'd list here but comments have a max length it seems??)