Knowledge Management

WMI:WinEventLog:Security - Discard events older than "x" months?

TylerTreat
Explorer

I've been able to start pulling AD logs via WMI which is nice and all, but I come in this morning and have 28 some odd million events in WMI:WinEventLog:Security. And a very unhappy splunk server after a long holiday weekend of chewing on events.

Is there a way to discard events past a certain age? We're still in trial mode for proof of concept and I'd like it to stay running a bit longer than a week.....

0 Karma

aaronkorn
Splunk Employee
Splunk Employee

in your inputs.conf you could add current_only = 1 and it should include only current events moving forward.

0 Karma

lukejadamec
Super Champion

Your WMI is collecting historic logs from the log folder.
If you have not already moved them, then it is probably too late because the data has already been indexed.

My recommendation is "don't worry about it". You will have an initial hit on indexing volume and performance, but once all of the old logs have been indexed you will have them for searching, or discarding as you see fit. If you have concerns about index volume, then you should call splunk support. As I recall there are ways to deal with initial license volume problems.

If you have a disk space problem, then you will need to remove the old data. This can be done with the index aging policy, but because that will be based on the most recent event on a bucket by bucket basis you may have problems because this is an inital data dump.

The buckets are where Splunk stores all of it's index data: splunk/var/lib/splunk/

WMI data is stored in the defaultdb by default. This folder will contain the buckets, and the bucket naming convention is "db_earliest event_epoch latest event epoch_unique ID". You can translate the epochs to time format with an epoch converter.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...