If you are polling events from the Windows EventLog via WMI there is an important setting that will correct this behavior for you. The setting is "current_only" which defaults to 0. Setting this to 1 instead will tell Splunk to only collect events that occur while Splunk is running.
Here is an example of "current_only" in use in a wmi.conf:
[WMI:LocalSecurity]
interval = 10
event_log_file = Security
index = default
disabled = 1
current_only = 1
See also:
http://www.splunk.com/base/Documentation/latest/Admin/Wmiconf
... View more