No events showing up for Splunk LWF

[lubinski]

here is how I have this test environment setup. Using vmware server I have 2 Windows XP Pro workstations setup with splunk. 1 is setup with as a receiver at 192.168.1.150:9997. The other machine is setup as a LWF, here is the outputs.conf;

[tcpout]
defaultGroup = 192.168.126.130_9997
disabled = false
indexAndForward=false


[tcpout:192.168.126.130_9997]
server = 192.168.1.150:9997

(the ip was changed thats why the group is wierd, but it still should work.)

All services are running, i can ping between the workstations, telnet also works but I'm not prompted for any login due to the lack of telnet on the receiving end.

Can someone help me troubleshoot this?

here is an excerpt from the splunkd logfile.

**********
03-06-2011 14:09:34.827 INFO  TcpOutputProc - Initializing with fwdtype=lwf
03-06-2011 14:09:34.827 INFO  WinEventLogChannel - Initialized Windows Event Log='System' Success; oldest_rec_id='1'; newest_rec_id='760'; total_rec='760'
03-06-2011 14:09:34.827 INFO  WinEventLogInputProcessor - main-thread: Processing existing Windows Event Log 'System'
03-06-2011 14:09:34.889 INFO  TcpOutputProc - Retrieving configuration from properties
03-06-2011 14:09:34.905 INFO  WinEventLogInputProcessor - main-thread: Finished processing existing Windows Event Log 'System': total_events='4' with empty_msg='0'.
03-06-2011 14:09:34.905 INFO  WinEventLogInputProcessor - main-thread: Starting to monitor Windows Event Log channels for events
03-06-2011 14:09:34.920 INFO  TcpOutputProc - found Whitelist forwardedindex.0.whitelist , RE : forwardedindex.0.whitelist
03-06-2011 14:09:34.920 INFO  TcpOutputProc - found Whitelist forwardedindex.1.blacklist , RE : forwardedindex.1.blacklist
03-06-2011 14:09:34.920 INFO  TcpOutputProc - found Whitelist forwardedindex.2.whitelist , RE : forwardedindex.2.whitelist
03-06-2011 14:09:34.920 INFO  TcpOutputProc - Will retry at max backoff sleep forever
03-06-2011 14:09:34.920 INFO  TcpOutputProc - Using clear text for server 192.168.1.150:9997
03-06-2011 14:09:34.920 INFO  TcpOutputProc - ALL Connections will use SSL with sslCipher=
03-06-2011 14:09:34.936 INFO  TcpOutputProc - initializing single connection with retry strategy for 192.168.1.150:9997
03-06-2011 14:09:34.936 INFO  loader - Instantiated plugin: controlqueueoutputprocessor
03-06-2011 14:09:34.936 INFO  PipelineComponent - Pipeline merging disabled in *mode.conf file
03-06-2011 14:09:34.936 INFO  PipelineComponent - Pipeline typing disabled in *mode.conf file
03-06-2011 14:09:34.936 INFO  loader - Running....
03-06-2011 14:09:34.936 INFO  PipelineComponent - Launching the pipelines.
03-06-2011 14:09:34.936 WARN  pipeline - Exiting pipeline distributedDeploymentNG gracefully: got eExit from processor distdeploymentNG
03-06-2011 14:09:34.936 INFO  TcpOutputProc - attempting to connect to 192.168.1.150:9997...
03-06-2011 14:09:34.936 WARN  IndexProcessor - received event for unconfigured/disabled index='_audit' with source='source::audittrail' host='host::user-58ecc80900' sourcetype='sourcetype::audittrail'
03-06-2011 14:09:34.936 WARN  pipeline - Exiting pipeline tail gracefully: got eExit from processor tail
03-06-2011 14:09:34.936 INFO  TcpOutputProc - Connected to 192.168.1.150:9997 
03-06-2011 14:09:34.936 WARN  pipeline - Empty pipeline (no processors): scheduler, exiting pipeline
03-06-2011 14:09:34.936 INFO  loader - Server supporting SSL v2/v3
03-06-2011 14:09:34.936 INFO  loader - Using cipher suite ALL:!aNULL:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM
03-06-2011 14:09:34.967 INFO  TPool - initializing BatchReaderTPool with 1 workers
*****

[lubinski]

I have found the answer. Upon creation of the VM's they both had the same name. I switched the name of one of them after splunk was installed but the C:\Program Files\Splunk\etc\system\local\inputs.conf still had the old hostname.

[lubinski]

I have now added;

[WinEventLog:Security]
disabled = 0
start_from = oldest
current_only = 0
checkpointInterval = 5

[WinEventLog:Application]
disabled = 0
start_from = oldest
current_only = 0
checkpointInterval = 5

[WinEventLog:System]
disabled = 0
start_from = oldest
current_only = 0
checkpointInterval = 5

to my C:\Program Files\Splunk\etc\apps\SplunkLightForwarder\default\inputs.conf and still no data on the server.

This shows up in the server splunkd log, maybe its the way I am searching for it? But the summary only shows one host.

03-07-2011 13:56:20.332 INFO TcpInputProc - Connection in cooked mode from 192.168.1.200 03-07-2011 13:56:20.332 INFO TcpInputProc - Registering metrics callback for: tcpin_connections 03-07-2011 13:56:20.332 INFO TcpInputProc - SSL clause not found or servercert not provided - SSL ports will not be available 03-07-2011 13:56:20.332 INFO TcpInputProc - port 9997 is reserved for splunk 2 splunk 03-07-2011 13:56:20.332 INFO TcpInputProc - using queueSize 1000 03-07-2011 13:56:20.364 INFO TcpInputProc - Valid signature found 03-07-2011 13:56:20.364 INFO TcpInputProc - Connection accepted from 192.168.1.200 03-07-2011 13:56:34.315 INFO TcpInputProc - Hostname=192.168.1.200 closed connection 03-07-2011 13:56:46.346 INFO TcpInputProc - Connection in cooked mode from 192.168.1.200 03-07-2011 13:56:46.346 INFO TcpInputProc - Registering metrics callback for: tcpin_connections 03-07-2011 13:56:46.346 INFO TcpInputProc - SSL clause not found or servercert not provided - SSL ports will not be available 03-07-2011 13:56:46.346 INFO TcpInputProc - port 9997 is reserved for splunk 2 splunk 03-07-2011 13:56:46.346 INFO TcpInputProc - using queueSize 1000 03-07-2011 13:56:46.346 INFO TcpInputProc - Valid signature found 03-07-2011 13:56:46.346 INFO TcpInputProc - Connection accepted from 192.168.1.200

[lubinski]

# Copyright (C) 2005-2010 Splunk Inc. All Rights Reserved. Version 4.1.7 # these here just override and disable stuff that in system/default. ####################################################################### # Turn these inputs off... they are on in system/default/inputs.conf #################################################################### [monitor://$SPLUNK_HOME\var\log\splunk\web_access.log] disabled = true [monitor://$SPLUNK_HOME\var\log\splunk\web_service.log] disabled = true [monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log] disabled = true [fschange:$SPLUNK_HOME\etc] disabled = true ################################ # Make sure these get forwarded ################################ [monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log] _TCP_ROUTING = * index = _internal # if you want metrics from lwf, create a new stanza for this in # SplunkLightForwarder\local\inputs.conf # note will cause indexing volume by host dashboard to be inaccurate # [monitor://$SPLUNK_HOME\var\log\splunk\metrics.log] # _TCP_ROUTING = * # index = _internal [monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log] _TCP_ROUTING = * index = _internal

[ftk]

It kind of looks like you don't have any inputs set up on your LWF. (Empty pipeline, no processors message in log). Can you post your inputs.conf please?