How do I trigger the re-indexing of events from a locally collected Windows Event Log channel?

[hexx]

I would like to force the re-indexing of events in a local Windows Event Log channel, let's say "Security". I have tried to clean the fishbucket but it had no effect on the Windows Event Log events. How can I do this?

[hexx · Accepted solution]

Splunk keeps track of what was read from Windows Event Log channel in checkpoint files. These files are "bookmark" flat text files that live in %SPLUNK_HOME%\var\lib\splunk\persistentstorage\WinEventLog. There is one file per monitored log channel :

C:\Program Files\Splunk\var\lib\splunk\persistentstorage\WinEventLog> dir
Volume in drive C is OS
Volume Serial Number is 1A2F-DE74

Directory of C:\Program Files\Splunk\var\lib\splunk\persistentstorage\WinEventLog

08/26/2011  09:12 AM    <DIR>          .
08/26/2011  09:12 AM    <DIR>          ..
06/24/2011  09:05 AM               152 c__Program_Files_Splunk_var_run_splunk_upload_application_evtx_checkpoint
06/24/2011  09:10 AM               134 C__Users_ledio_Desktop_test_application_evtx_checkpoint
08/11/2011  12:23 PM               103 Security_checkpoint
08/11/2011  12:11 PM                94 Setup_checkpoint
08/11/2011  12:11 PM                96 System_checkpoint
           5 File(s)            579 bytes
           2 Dir(s)  132,161,089,536 bytes free

Contents of Security_checkpoint :

<BookmarkList>
  <Bookmark Channel='Security' RecordId='319739723' IsCurrent='true'/>
</BookmarkList>

In order to force the re-indexing of all available events for a given channel, one simply needs to delete the corresponding checkpoint file and restart splunkd. It is possible to fiddle with the RecordId field to re-index from a given event number, but this is usually harder to figure out.

[wrangler2x]

I'm not finding the checkpoint files on a splunk 6.14 forwarder that is sending me Windows Event Logs. Have they moved?

[Jeff_Lightly_Sp]

I'm seeing them at: Program Files-SplunkUniversalForwarder-var-lib-splunk-modinputs-WinEventLog

[pavankumarh]

i was fed up deleting the fishbucket multiple times and using the btprobe. 

deleting the "application" file under "\SplunkUniversalForwarder\var\lib\splunk\modinputs\WinEventLog" did the job. thank you. 

[hexx]

@hjohnson : Would you share with us your inputs.conf configuration stanzas that set up the Event Log channel inputs?

[hjohnson]

Yes... I checked on the server that reads directly from the Event Log Channels. I do not have any remote forwarders.

[hexx]

@hjohnson : Are you certain that you checked on the server that reads directly from the Event Log channels? I suspect that you may have checked the indexer, when the event logs are collected by a remote forwarder, in which case you'll have to perform this operation on the forwarder itself.

[hjohnson]

This answer does not work for me because that directory (C:\Program Files\Splunk\var\lib\splunk\persistentstorage\WinEventLog) does not exist and there are no files with "_checkpoint" in the file system except for wmi_checkpoint. And deleting that file and restarting Splunk does not seem to get the job done.

[johnt0]

Try: C:\Program Files\SplunkUniversalForwarder\var\lib\splunk\persistentstorage\WinEventLog
or C:\Program Files (x86) SplunkUniversalForwarder ...

by default the SplunkUniversalForwarder is where splunk lives i think