Getting Data In

Domain Controller logs - Best practice?

Path Finder

We are currently pulling the event logs for 6-8 domain controllers.
We are having issues with some of the domain controllers as it seems it can't handle the volume, and isn't updating for 6,7 hours when it should be updating every 30 minutes.

Are there any best practices for this? Has anyone experienced similar behaviour?

0 Karma


The security logs can be quite voluminous on a busy AD server.

First, you say you are "pulling" the logs. If you are using WMI, you will probably have far better results using the Universal Forwarder on each. WMI is better than it used to be, but it doesn't hold a candle to the UF for being efficient. So if it is the case that you are using WMI, I'd start by installing the UF and configuring it to send in what you want.

After that change, if necessary, or if you are already using the UF, then I'd check how much CPU, RAM and Disk IO are in use by the servers both with and without the UF. You may be pushing the edge of the servers capabilities regardless of Splunk, and in that case you either need to increase the specs for the servers or add more AD servers. If this is happening only on a few, perhaps they are under spec. Or if they're in their own site, maybe it's only one site that needs another AD server or two.

Lastly, you can tune the UF a bit - I'm not sure it really cuts down on the server's load but it's worth a try to look at the events coming in. If you can identify large chunks of those that you don't need it COULD help to blacklist them [Note 1]. This won't really help much until you've done both of the above - making sure you are using the UF and making sure the servers aren't too heavily loaded already.

[1] Some group policy notifications come to mind - search your Splunk over the past hour with something like ... | stats count by EventCode and look closely at the top 3 or 4 and see if you need them for anything.

0 Karma


Oh, to be specific, best practices:

1) Use the UF, not WMI (especially on busier servers).
2) Make sure the server has enough free capacity to continue doing AD and also add the UF's load to it.
3) Only collect what you are going to use (or can reasonably see using in the future). Domain Servers are Mrs. Chatty Cathy, but you don't really need all of it.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!