Getting Data In

How do I trigger the re-indexing of events from a locally collected Windows Event Log channel?

Splunk Employee
Splunk Employee

I would like to force the re-indexing of events in a local Windows Event Log channel, let's say "Security". I have tried to clean the fishbucket but it had no effect on the Windows Event Log events. How can I do this?

1 Solution

Splunk Employee
Splunk Employee

Splunk keeps track of what was read from Windows Event Log channel in checkpoint files. These files are "bookmark" flat text files that live in %SPLUNK_HOME%\var\lib\splunk\persistentstorage\WinEventLog. There is one file per monitored log channel :

C:\Program Files\Splunk\var\lib\splunk\persistentstorage\WinEventLog> dir
Volume in drive C is OS
Volume Serial Number is 1A2F-DE74

Directory of C:\Program Files\Splunk\var\lib\splunk\persistentstorage\WinEventLog

08/26/2011  09:12 AM    <DIR>          .
08/26/2011  09:12 AM    <DIR>          ..
06/24/2011  09:05 AM               152 c__Program_Files_Splunk_var_run_splunk_upload_application_evtx_checkpoint
06/24/2011  09:10 AM               134 C__Users_ledio_Desktop_test_application_evtx_checkpoint
08/11/2011  12:23 PM               103 Security_checkpoint
08/11/2011  12:11 PM                94 Setup_checkpoint
08/11/2011  12:11 PM                96 System_checkpoint
           5 File(s)            579 bytes
           2 Dir(s)  132,161,089,536 bytes free

Contents of Security_checkpoint :

<BookmarkList>
  <Bookmark Channel='Security' RecordId='319739723' IsCurrent='true'/>
</BookmarkList>

In order to force the re-indexing of all available events for a given channel, one simply needs to delete the corresponding checkpoint file and restart splunkd. It is possible to fiddle with the RecordId field to re-index from a given event number, but this is usually harder to figure out.

View solution in original post

Splunk Employee
Splunk Employee

Splunk keeps track of what was read from Windows Event Log channel in checkpoint files. These files are "bookmark" flat text files that live in %SPLUNK_HOME%\var\lib\splunk\persistentstorage\WinEventLog. There is one file per monitored log channel :

C:\Program Files\Splunk\var\lib\splunk\persistentstorage\WinEventLog> dir
Volume in drive C is OS
Volume Serial Number is 1A2F-DE74

Directory of C:\Program Files\Splunk\var\lib\splunk\persistentstorage\WinEventLog

08/26/2011  09:12 AM    <DIR>          .
08/26/2011  09:12 AM    <DIR>          ..
06/24/2011  09:05 AM               152 c__Program_Files_Splunk_var_run_splunk_upload_application_evtx_checkpoint
06/24/2011  09:10 AM               134 C__Users_ledio_Desktop_test_application_evtx_checkpoint
08/11/2011  12:23 PM               103 Security_checkpoint
08/11/2011  12:11 PM                94 Setup_checkpoint
08/11/2011  12:11 PM                96 System_checkpoint
           5 File(s)            579 bytes
           2 Dir(s)  132,161,089,536 bytes free

Contents of Security_checkpoint :

<BookmarkList>
  <Bookmark Channel='Security' RecordId='319739723' IsCurrent='true'/>
</BookmarkList>

In order to force the re-indexing of all available events for a given channel, one simply needs to delete the corresponding checkpoint file and restart splunkd. It is possible to fiddle with the RecordId field to re-index from a given event number, but this is usually harder to figure out.

View solution in original post

Motivator

I'm not finding the checkpoint files on a splunk 6.14 forwarder that is sending me Windows Event Logs. Have they moved?

0 Karma

Communicator

I'm seeing them at: Program Files-SplunkUniversalForwarder-var-lib-splunk-modinputs-WinEventLog

Splunk Employee
Splunk Employee

@hjohnson : Would you share with us your inputs.conf configuration stanzas that set up the Event Log channel inputs?

0 Karma

New Member

Yes... I checked on the server that reads directly from the Event Log Channels. I do not have any remote forwarders.

0 Karma

Splunk Employee
Splunk Employee

@hjohnson : Are you certain that you checked on the server that reads directly from the Event Log channels? I suspect that you may have checked the indexer, when the event logs are collected by a remote forwarder, in which case you'll have to perform this operation on the forwarder itself.

0 Karma

New Member

This answer does not work for me because that directory (C:\Program Files\Splunk\var\lib\splunk\persistentstorage\WinEventLog) does not exist and there are no files with "_checkpoint" in the file system except for wmi_checkpoint. And deleting that file and restarting Splunk does not seem to get the job done.

0 Karma

Engager

Try: C:\Program Files\SplunkUniversalForwarder\var\lib\splunk\persistentstorage\WinEventLog
or C:\Program Files (x86) SplunkUniversalForwarder ...

by default the SplunkUniversalForwarder is where splunk lives i think

0 Karma