How to monitor Windows Event Logs that roll to an archive every hour?

[karlbosanquet]

I have a WinEventLog://System log which rolls to archive every hour or so. I have 4 questions;

1) is the Splunk Universal Forwarder (UF) clever enough to ingest archived files based on the default [WinEventLog://System] input or does it only ingest the data in the current log?

2) Does the UF catch all events in the log or is there a chance some events could be lost at the point when the log rolls?

3) if either the UF or Index layer are unavailable for a period of time (possibly days), will all of the logs be lost until the connection is re-established?

4) what is Splunk's recommended optimum file size for a WinEventLog source?

[Richfez]

1) I believe it only retrieves data from the current log.
2) I doubt it'll miss anything. The way it hooks into the event logs is pretty robust. I guess it stands to reason I should ask how your logs roll to an archive? On my windows boxes, Splunk IS my archive for logs. Windows is constantly rolling the back end off the logs and overwriting events as required (on some of my DCs at certain times I only have an hour's worth of logs on the box). It handles this perfectly.
3) You can configure a persistent queue to make sure you cover your storage needs.
4) Not sure I understand this question. If you are referring to a cache file, you'd have to figure that out yourself by looking at the size of your X event log, then deciding how long of an outage you want to handle and set it appropriately. For instance, if it's 100 MB, rolls once and hour and you want to be sure to keep 24 hours in a cache, then you'll need 100*24, or about 2.4 GB of persistent queue. Maybe make that 3 GB to cover unexpected volume...

I hope that helps. Happy Splunking!
-Rich

[karlbosanquet]

1) Ok, I have spoken to Splunk to confirm if this is the case or not.
2) When a WinEventLog is archived, it renames the whole file and creates a new file for new events. The question was more around how Splunk UF handles this, or could an event(s) be missed when the file is renamed.
3) will do, I will also look at ingesting the archived evtx as an option.
4) As the UF needs to read the whole event log for processing, what is the optimum size for the windows event log to be. Max is 4GB, however it takes ages to load the log when it is nearly full. Too small file and it will roll quickly. What is the 'optimum' size of a Windows Event log for the Splunk Universal Forwarder.