WMI:WinEventLog:Security - Discard events older than "x" months?


I've been able to start pulling AD logs via WMI which is nice and all, but I come in this morning and have 28 some odd million events in WMI:WinEventLog:Security. And a very unhappy splunk server after a long holiday weekend of chewing on events.

Is there a way to discard events past a certain age? We're still in trial mode for proof of concept and I'd like it to stay running a bit longer than a week.....

0 Karma

Splunk Employee
Splunk Employee

in your inputs.conf you could add current_only = 1 and it should include only current events moving forward.

0 Karma

Super Champion

Your WMI is collecting historic logs from the log folder.
If you have not already moved them, then it is probably too late because the data has already been indexed.

My recommendation is "don't worry about it". You will have an initial hit on indexing volume and performance, but once all of the old logs have been indexed you will have them for searching, or discarding as you see fit. If you have concerns about index volume, then you should call splunk support. As I recall there are ways to deal with initial license volume problems.

If you have a disk space problem, then you will need to remove the old data. This can be done with the index aging policy, but because that will be based on the most recent event on a bucket by bucket basis you may have problems because this is an inital data dump.

The buckets are where Splunk stores all of it's index data: splunk/var/lib/splunk/

WMI data is stored in the defaultdb by default. This folder will contain the buckets, and the bucket naming convention is "dbearliest eventepoch latest event epoch_unique ID". You can translate the epochs to time format with an epoch converter.

0 Karma