Windows Server with Corrupted Security Log

taylorchase — Fri, 04 Mar 2011 05:48:42 GMT

I have a server that had a corrupted Security Log.

In order to resolve that problem I backed up the security log and cleared it.

Now new events are not showing up from that server's security log in splunk. I did some checking of the _internal index and saw that the wmi query that is being used is checking for items where RecordNumber > 6918747. When I cleared the event log, it reset the RecordNumber to 1 on the server, and since 6 million+ security items have not yet happened, the wmi query is gathering no information from that server.

Is there a command I can run to reset this remembered checkpoint value?

Thanks

Re: Windows Server with Corrupted Security Log

hexx — Wed, 16 Mar 2011 07:53:51 GMT

Hello Taylor,

Splunk records it's current "position" when reading remote event logs in the %SPLUNK_HOME%\var\lib\splunk\persistentstorage\wmi_checkpoint file.

This file is in SQLite format and keeps track of Splunk's position in the remote event log stream by using the events' RecordNumber.

Here is what entries in the file look like when exported to CSV using an SQLite viewer :


"primarykey","secondary1","secondary2","secondary3","value"
"\ruraljuror\root\cimv2|3cedb797","","","","D 6918747 6918746 6918747"
"\ruraljuror\root\cimv2|cc2f7ffa","","","","D 824 823 825 20080407195557.000000-240"
"\ruraljuror\root\cimv2|9a9cabe3","","","","D 172 171 172 20080408141453.000000-240"
"\ruraljuror\root\cimv2|3d8d2a65","","","","D 1853 1852 1853 20080408192617.000000-240"
"\ruraljuror\root\cimv2|16e0ea0e","","","","D 18624 18623 18625"
"\ruraljuror\root\cimv2|22c75216","","","","D 57280 57279 57280"
"\ruraljuror\root\cimv2|cee114bd","","","","D 67780 67779 67781"

Note that the event log channels are hashed by as the CRC32 checksum of the channel name.

Example :

Application -> 22C75216
System -> CEE114BD
Security -> 3CEDB797

To look for a specific channel entry, just run the channel name through a CRC32 checksum generator such as http://crc32-checksum.waraxe.us and look for that entry in the database.

As an example, let's say that we want to trigger the re-indexing of all event logs for channel "Security" from host "ruraljuror".

Using an SQLite editor, we would simply drop the following row from the table :


"\ruraljuror\root\cimv2|3cedb797","","","","D 6918747 6918746 6918747"

NOTE : This change should be made when Splunk is stopped.

With that entry removed, Splunk should re-index all "Security" event logs from host "ruraljuror" on it's next restart.

topic Windows Server with Corrupted Security Log in Getting Data In

Windows Server with Corrupted Security Log

Re: Windows Server with Corrupted Security Log