Running a Windows 2012 R2 DHCP Server with UF 9.0.1 and Splunk Enterprise 8.0.5. My inputs at the UF look like this:
[default]
index = windowsdhcp
_TCP_ROUTING = prod
[WinEventLog://System]
start_from = oldest
disabled = 0
current_only = 0
whitelist1 = SourceName="DhcpServer"
whitelist2 = SourceName="Dhcp-Server"
[WinEventLog://DHCPAdminEvents]
start_from = oldest
disabled = 0
My issue is that the whitelisted events in the 1st stanza are not getting processed to the indexer. If I review the XML of the events in the Windows Event Viewer:
These events are collected and indexed:
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
<Provider Name="Microsoft-Windows-DHCP-Server" Guid="{6D64F02C-A125-4DAC-9A01-F0555B41CA84}" />
<EventID>20251</EventID>
<Version>0</Version>
<Level>4</Level>
<Task>121</Task>
<Opcode>106</Opcode>
<Keywords>0x2000000000000000</Keywords>
<TimeCreated SystemTime="2022-10-29T12:25:40.655052000Z" />
<EventRecordID>161</EventRecordID>
<Correlation />
<Execution ProcessID="3884" ThreadID="4472" />
<Channel>DhcpAdminEvents</Channel>
<Computer>dhcp-srv-a.mydomain.com</Computer>
<Security UserID="S-1-5-20" />
</System>
- <EventData>
<Data Name="Server">dhcp-srv-b.mydomain.com</Data>
<Data Name="RelationName">dhcp-srv-b.mydomain.com-dhcp-srv-a.mydomain.com</Data>
<Data Name="OldState">COMMUNICATION_INT</Data>
<Data Name="NewState">NORMAL</Data>
</EventData>
</Event>
These events do not get captured (Note: event is in classic format):
Log Name: System
Source: Microsoft-Windows-DHCP-Server
Date: 14/11/2022 23:11:37
Event ID: 1376
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: dhcp-srv-a.mydomain.com
Description:
IP address range of scope 10.119.6.0 is 89 percent full with only 6 IP addresses available.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-DHCP-Server" Guid="{6D64F02C-A125-4DAC-9A01-F0555B41CA84}" EventSourceName="DhcpServer" />
<EventID Qualifiers="0">1376</EventID>
<Version>0</Version>
<Level>3</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2022-11-14T23:11:37.000000000Z" />
<EventRecordID>87097</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>System</Channel>
<Computer>dhcp-srv-a.mydomain.com</Computer>
<Security />
</System>
<EventData>
<Data>10.119.6.0</Data>
<Data>89</Data>
<Data>6</Data>
</EventData>
</Event>
I can't see why it is not collecting the second event via the 1st stanza?
I finally figured it out! So I suspected that the UF thought it had already collected these events so I had a look into this file %SPLUNK_HOME %\var\lib\splunk\modinputs\WinEventLog\System which basically contains a record where the UF left off on a given WinEventlog i.e. the EventID it last consumed.
<BookmarkList>
<Bookmark Channel='System' RecordId='1236992' IsCurrent='true'/>
</BookmarkList>
Event 1236992 is one higher than the last instance of the events I wish to whitelist and consume (you can see the EventID in the windows event viewer). I stepped through the following:
I now have all my events!
Note: this may contain duplicates events so we can use delete to remove them based on EventID. I then deleted other events form my index that I did not require via https://docs.splunk.com/Documentation/SplunkCloud/latest/SearchReference/Delete. Be careful if running this!
The plot thickens! So the sourcename was actually Microsoft-Windows-DHCP-Server. This is as per the copy/paste to notepad of the XML view in Widows event veiwer and shown above as
Log Name: System
Source: Microsoft-Windows-DHCP-Server
Date: 14/11/2022 23:11:37
Event ID: 1376
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: dhcp-srv-a.mydomain.com
Description:
IP address range of scope 10.119.6.0 is 89 percent full with only 6 IP addresses available.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-DHCP-Server" Guid="{6D64F02C-A125-4DAC-9A01-F0555B41CA84}" EventSourceName="DhcpServer" />
<EventID Qualifiers="0">1376</EventID>
<Version>0</Version>
<Level>3</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2022-11-14T23:11:37.000000000Z" />
<EventRecordID>87097</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>System</Channel>
<Computer>dhcp-srv-a.mydomain.com</Computer>
<Security />
</System>
<EventData>
<Data>10.119.6.0</Data>
<Data>89</Data>
<Data>6</Data>
</EventData>
</Event>
I updated my input stanza as follows and now working :
[WinEventLog://System]
start_from = oldest
disabled = 0
current_only = 0
whitelist1 = SourceName="Microsoft-Windows-DHCP-Server"
I basically pushed all events from the System event log to my testSplunk instance and saw what SourceName it pulled out of the classic event.
That said, I am now still getting ALL events frm the System log! My entire inputs.conf file is as follows
[default]
index = windowsdhcp
_TCP_ROUTING = ssl-production
[WinEventLog://System]
start_from = oldest
disabled = 0
current_only = 0
whitelist = SourceName="Microsoft-Windows-DHCP-Server"
[WinEventLog://DHCPAdminEvents]
start_from = oldest
disabled = 0
[WinEventLog://Microsoft-Windows-Dhcp-Server/Operational]
start_from = oldest
disabled = 0
[monitor://$WINDIR\System32\DHCP]
disabled = 0
whitelist = DhcpSrvLog*
crcSalt = <SOURCE>
sourcetype = DhcpSrvLog
So I then change my inputs to this to see if I can stop any system events coming whatsoever!
[default]
index = windowsdhcp
_TCP_ROUTING = ssl-production
[monitor://$WINDIR\System32\DHCP]
disabled = 0
whitelist = DhcpSrvLog*
crcSalt = <SOURCE>
sourcetype = DhcpSrvLog
But they keep on coming even though the change is present and btool output does not show any entries for WinEventLog://System
I had to restart the universal forwarder so it to stop picking up all System event log events. On anoher DHCP server I am back to square one with the original monitoring stanza not working with the whitelisting
I'm really stumped as to what is going on here! It#s almost like current_only has no effect 😞
I finally figured it out! So I suspected that the UF thought it had already collected these events so I had a look into this file %SPLUNK_HOME %\var\lib\splunk\modinputs\WinEventLog\System which basically contains a record where the UF left off on a given WinEventlog i.e. the EventID it last consumed.
<BookmarkList>
<Bookmark Channel='System' RecordId='1236992' IsCurrent='true'/>
</BookmarkList>
Event 1236992 is one higher than the last instance of the events I wish to whitelist and consume (you can see the EventID in the windows event viewer). I stepped through the following:
I now have all my events!
Note: this may contain duplicates events so we can use delete to remove them based on EventID. I then deleted other events form my index that I did not require via https://docs.splunk.com/Documentation/SplunkCloud/latest/SearchReference/Delete. Be careful if running this!
Hi @shocko,
The first Event is:
<Channel>DhcpAdminEvents</Channel>
So this one is processed by your DHCPAdminEvents Stanza. There is no white/blacklist, so it is processed.
To allow the second Event, I guess the whitelist entry has to be adjusted.
I did not use this recently, but looking at the docs it has to be key=regex syntax.