Best method for pulling Microsoft DNS logs with Splunk?

[tgow]

What is the best method for pulling Windows DNS Logs with Splunk. I am looking at the following methods:

1. Send directly via syslog

2. Send the to SCOM then have Splunk read the SCOM logs with a Forwarder

3. Enable the creation of a DNS debug file

Thanks in advance.

[woodcock]

These answers are all old and nowadays almost nobody gets DNS events from a Windows server from the logs, the smart way is to pull them off the wire with stream. Trust me: you will regret trying to do any correlations with the app logs but it will all be a BREEZE with stream:

http://www.rfaircloth.com/2015/11/06/get-started-with-splunk-app-stream-6-4-dns/

[vikkysplunk]

@woodcock Here i have one doubt, these stream TA we need to deploy on only DNS servers or all the windows servers in environment.

Thanks in advance.

[woodcock]

DNS servers.

[keithevanscdcr]

If my enterprise AD admins will not allow Splunk Agent on DCs, is DNS Debug logging and Windows Event Forwarding my only option?  Do you have any reference/suggestions regarding this configuration?  Additionally, and read on the article, it seems Stream is the preferred implementation and not DNS Debug. Can you elaborate as to why?

Thank you much in advance.

[rnagheereddy]

In my environment we needed to capture all the DNS queries made by user's PCs against the Windows AD DC DNS servers but ignore any queries for our own domains eg *.company.com, *.ad.company.com etc. (Our Windows DNS servers are authoritative for only the ad.company.com domain, they "forward" queries for all other domains.) We did not need to monitor queries against the ad.company.com zone - too much junk - so we didn't want to forward this useless data to our Splunk Indexers.

There is a special setting you must configure to ensure that the DNS log file can be monitored:
Use this command:

dnscmd MyDNSSRV /config /logLevel 0x8000e101

(cf http://technet.microsoft.com/en-us/library/cc772069(WS.10).aspx)

Our solution:

1. Enable DNS debug logging on the DCs. Choose to capture only the incoming queries.
2. Install Splunk Heavy Forwarder on the DC.
3. Configure the Forwarder to monitor the DNS log file.
4. Configure transforms.conf and props.conf on the Forwarder to filter out (drop) the undesired queries.

There is an increased CPU load on the DC (from the debug logging and the filtering of the events) so YMMV. We had sufficient capacity.

$SPLUNK_HOME\etc\apps\launcher\local\props.conf

[win_dns]
TRANSFORMS-drop = dropline

$SPLUNK_HOME\etc\apps\launcher\local\transforms.conf

[dropline]
REGEX = \(9\)[Cc][Oo][Mm][Pp][Aa][Nn][Yy]\(3\)[Cc][Oo][Mm]
DEST_KEY = queue
FORMAT = nullQueue

$SPLUNK_HOME\etc\apps\launcher\local\inputs.conf

[monitor://c:\dnslogs\wind_dns.csv]
disabled = false
followTail = 1
sourcetype = win_dns
index = win_dns

[rnagheereddy]

Megan - you're right. I must've been in a hurry.
We actually use the value of:
0x8000e101

[megancarney]

This worked for us, with a slight modification. In the command:
dnscmd MyDNSSRV /config /logLevel 0x6101
"0x6101" probably won't get you much.

So just make sure the hex value you put in reflects the options you will choose when you enable DNS debug logging in step 1.

[Mannyi31]

I have implemented different approaches on how to do just this and the best one that has worked without giving me problems is the universal Forwarder method. If you are going to use this method you will need to enable the creation of DNS debug file on the local server (anyware on the server is fine as long as you got enough space) and configure the universal forwarder during installation to monitor the DNS debug file and send the data to the Indexer on the port that you chose.
This method is recommended not just because the forwarder is keeping track of the data as mentioned but this has the ability to monitor other types of event logs and forward them using the same forwarder in case that you want more than just DNS logs.

[Mannyi31]

I have not done this but looking around I found this article talking about doing exacly what you are trying to do. It is for an older version of Splunk (4.1.3) but it is usefull:

http://splunk-base.splunk.com/answers/4546/field-extraction-regex-fu-help

Also check on this link for the updated information on SEDCMD, REGEX and SED:

http://docs.splunk.com/Documentation/Splunk/4.2.5/Data/Anonymizedatausingconfigurationfiles

Basically this is used to anonymize confidential data from the logs and can be used to replace values with different ones like what you are trying to do

[ageld]

I can get rid of (\d+) stuff with the following statements in search:
sourcetype="DNSDebugLog" | eval dns_name=replace(dns_name,"(\d+)",".") | eval dns_name=replace(dns_name,"^.","") | table dns_name

but I do not those to appear in the log at all. I want to replace those on the forwarder before the logs are sent to the indexer

[ageld]

Mannyi31, have you figured out how to get rid of (\d+) in dns names of debug file log entries:

(3)dns(8)msftncsi(3)com(0)
(3)www(16)google-analytics(3)com(0)

I would like they to appear as:
dns.msftncsi.com
www.google-analytics.com

I want prepending (\d+) to be replaced with nothing and the other ones to be replaced with dots except the trailing one.

I've figured out how to extract DNS names from the logs:

(?i)] \w+\s+(?P(.+))

but I am puzzled how to do post-processing to get rid of those numbers in parenthesis. My guess it has to be done in transforms.conf file.

[sdwilkerson]

What version of Windows server? It makes a difference.

[_d_ · Accepted solution]

Best recommended method is to persist your data to disk and then have a Forwarder monitor it. Sending it via Syslog may be prone to errors due network problems and/or when an Indexer is down, for whatever reason, including maintenance. Forwarders will keep track of what has been sent for indexing, something that syslog or any other network forwarding methods are not capable of (this, among other things, reduces the risk of having duplicate data in your indexes).

Hope this helps.

> please upvote and accept answer if you find it useful - thanks!