topic Re: Windows DNS debug logs TA not using CIM fields? in All Apps and Add-ons

Windows DNS debug logs TA not using CIM fields?

asieira — Wed, 20 May 2015 18:14:36 GMT

I set up a test environment with a Windows 2012 R2 server with the DNS server role, and was able to successfully install TA-DNSServer-NT6 (as per the instructions at http://docs.splunk.com/Documentation/MSApp/1.1.2/MSInfra/DownloadandconfiguretheSplunkAdd-onsforWindowsDNS ).

However, I was very surprised to discover two things:

The DNS packet data (which contains the actual replies, such as the IP addresses a hostname resolved to in a A query) was not being parsed, and in fact generated "incomplete" alerts with each packet line as in this example:

The summary lines were being correctly parsed, but the corresponding CIM fields (http://docs.splunk.com/Documentation/CIM/4.2.0/User/NetworkResolutionDNS ) were not extracted. For example, a query for tag=dns came up empty even though there are DNS events and the Splunk App for Windows Infrastructure "DNS: Top Requested Queries" report is being populated. Here is an example:

For one, I would have expected to see tags with network, resolution and dns values as per the CIM documentation.

Can anyone else with a working installation of the DNS debug log collection confirm whether this field is populated for them? Did I make some mistake in the setup or is this a limitation of the app?

Re: Windows DNS debug logs TA not using CIM fields?

asieira — Wed, 24 Jun 2015 14:59:42 GMT

So it turns out that the latest available version of the app claims support to CIM 4.0. And the Network Resolution (DNS) data model didn't show up until CIM 4.1.0 was released around 6 months ago.

So apparently it's a documented current limitation of the app. Oh, well.

Re: Windows DNS debug logs TA not using CIM fields?

mikaelbje — Wed, 19 Aug 2015 10:11:47 GMT

Perhaps time for someone to update the app ...

Re: Windows DNS debug logs TA not using CIM fields?

mikaelbje — Tue, 29 Sep 2020 07:02:35 GMT

Updated 2019-11-27 to account for _carddavs_.tcp.(null) possibility.
Tested with Splunk Add-on for Windows 7.0.0 and Splunk version 8.0.

I'm answering this thread because I will eventually need this myself.

The current limitations I've identified that I'm attempting to fix:

CIM compliance
Normalization of the query name to get rid of the (4)imap(5)gmail(3)com(0) ugliness
IPv6 addresses are not populated in src

Add these additions to your Splunk_TA_windows directory. You need to deploy this to your Indexer and Search Head.

local/tags.conf:

[eventtype=nt6-dns-events]
dns = enabled
network = enabled
resolution = enabled

* local/props.conf:

[MSAD:NT6:DNS]
# Index-time Normalization of the query name
# Example 1: (3)www(6)google(3)com(0) becomes www.google.com
# Example 2: (9)_carddavs(4)_tcp(6)(null)(0) becomes _carddavs._tcp(null)
#
SEDCMD-win_dns = s/(\\(\d+\\))/./g s/(\\.)$// s/\s(\\.)/ /
# I don't know if MS DNS is Query or Response, so I set both
EVAL-message_type = mvappend("QUERY","RESPONSE")
FIELDALIAS-message_type = questiontype AS record_type
FIELDALIAS-vendor_query_type = opcode AS vendor_query_type
LOOKUP-dns_query_type_lookup = msad_dns_vendor_query_type_lookup vendor_query_type OUTPUT query_type

* local/transforms.conf:

# Reference: tools .ietf.org/html/rfc2929#section-2.2 
[msad_dns_vendor_query_type_lookup]
filename = dns_vendor_query_type_lookup.csv
# IP version agnostic src
[KV_for_RecvdIP]
REGEX = (Snd|Rcv)\s+((?:(?:\d{1,3}\\.){3}(?:\d{1,3}))|(?:(?:::)?(?:[\dA-Fa-f]{1,4}:{1,2}){1,7}(?:[\d\%A-Fa-z\\.]+)?(?:::)?)|(?:::[\dA-Fa-f\\.]{1,15})|(?:::))
FORMAT = src::$2
#
[KV_for_Domain]
REGEX = \\.([a-zA-Z0-9\\-\\_]+\\.[a-zA-Z0-9\\-\\_]+)$
FORMAT = src_domain::$1

lookups/dns_vendor_query_type_lookup.csv:

vendor_query_type,query_type
Q,Query
N,Notify
U,Update
?,unknown

Re: Windows DNS debug logs TA not using CIM fields?

asieira — Thu, 20 Aug 2015 13:34:38 GMT

Turns out that fixing the "ugly" hostnames as generated in Windows DNS debug logs is unfortunately not as simple as the regex replace you implemented there.

First of all, sometimes they contain pointers in brackets that we also need to get rid of: http://stackoverflow.com/questions/20381717/windows-dns-server-debug-log-hostname-format

Second, parenthesis are valid characters inside domain/host names. So it might be that a valid occurrence of a number inside parenthesis will be incorrectly replaced by a dot. One example of an actual use of parenthesis (though not with numbers inside) that is rather common in Windows networks is the following: (9)_carddavs(4)_tcp(6)(null)(0) which should be converted to _carddavs._tcp.(null).

So the only viable solution that will always work is to actually write code that treats the numbers in parenthesis as what they really are - the length of each dot-separated string that makes up the domain/host name in question. Maybe an external command implementation in Python is in order here.

Re: Windows DNS debug logs TA not using CIM fields?

mreynov_splunk — Fri, 21 Aug 2015 14:34:43 GMT

we filed a bug and this will be prioritized into the dev queue. thanks for bringing this to our attention!

Re: Windows DNS debug logs TA not using CIM fields?

mikaelbje — Mon, 24 Aug 2015 11:46:04 GMT

Bummer. I'd never seen a DNS record with a (null) value before, so discard the SEDCMD-win_dns = s/\(\d+\)/./g attribute from the props.conf. I wonder how Enterprise Security Suite reacts to these funky looking records in Windows DNS logs.

Re: Windows DNS debug logs TA not using CIM fields?

fharding — Tue, 29 Sep 2020 07:51:04 GMT

Mikealbje, I just implemented this workaround in our Splunk instance. Thank you! This just saved me hours of work!

BTW, I had to fix a typo - the stanza in transforms.conf:
[msad_dns_vendor_query_type_lookup]
should be:
[dns_vendor_query_type_lookup]

Another modification I made to your method was copying the default DNSServer-NT6 TA to a custom TA "DNSServer-NT6_cimfix", implementing your fixes, and deleting the default/inputs.conf. I have the original vanilla DNSServer-NT6 TA pushing down to the AD boxes for log collection, and the DNSServer-NT6_cimfix band-aid version of the TA pushing to the search head and indexer for index-time and search-time field extraction.

The extractions look like they're working. Just waiting on Network Sessions (DNS) data model acceleration to rebuild and the DNS ES dashboard should be open for business.

Re: Windows DNS debug logs TA not using CIM fields?

fharding — Tue, 10 Nov 2015 00:26:44 GMT

Darn - I spoke too soon. Looks the searches behind the panels in the ES dashboard want more fields. Going to see how much work will be involved in normalizing this data source further. Best option might just be to let the Splunk devs do the work and wait for an official update to the TA.

Re: Windows DNS debug logs TA not using CIM fields?

mikaelbje — Tue, 10 Nov 2015 05:31:41 GMT

Btw there's a new community add-on for Windows DNS logs. It uses a new method to get the logs and claims to be CIM compliant. I haven't been able to test it yet. Look it up on Splunkbase and let me know if it works 😉

Re: Windows DNS debug logs TA not using CIM fields?

asieira — Tue, 10 Nov 2015 11:19:37 GMT

Do you have a link for it? Might be helpful for people that read this thread later on.

Re: Windows DNS debug logs TA not using CIM fields?

fharding — Tue, 10 Nov 2015 17:11:16 GMT

Are you talking about this one?

https://splunkbase.splunk.com/app/2937/

If so, I'm curious if it works with Server 2008 flavors. Says it's built for 2012r2 and up.

Re: Windows DNS debug logs TA not using CIM fields?

mikaelbje — Wed, 11 Nov 2015 05:26:35 GMT

That's the one. Untested by me, but claims to be CIM compliant.

Re: Windows DNS debug logs TA not using CIM fields?

asieira — Wed, 11 Nov 2015 11:58:44 GMT

That might be an alternative, yes. Just keep in mind it's not using the DNS debug logs though, instead it uses new DNS logging functionality Microsoft introduced in 2012r2.

Re: Windows DNS debug logs TA not using CIM fields?

Richfez — Wed, 11 Nov 2015 13:17:49 GMT

For many of these reasons we didn't use this method at all but instead deployed the Splunk App for Stream and used that to sniff DNS packets from the DNS servers. This is also how we're getting DHCP packets in.

It may be an option for you as well. It is a bit of work getting it set up and running, but far less work than you've already put into fixing the DNS logs from MS. Because we're selectively only pulling in DNS and DHCP, I believe it's the same license hit as using the logs would be.

Note: a prerequisite is having winpcap libraries installed because the stream app uses winpcap on the clients to actually sniff the packets, and IIRC the docs aren't as clear as they could be on that. But, that's a small, fast, easy install.

Re: Windows DNS debug logs TA not using CIM fields?

asieira — Fri, 11 Mar 2016 23:57:16 GMT

Any updates on this?

Re: Windows DNS debug logs TA not using CIM fields?

asieira — Fri, 05 Aug 2016 12:52:22 GMT

@rich7177 is there any walkthrough or tutorial posted anywhere on how to collect DNS data using Splunk App for Stream? Also, would you mind sharing what the DNS events look like (extracted fields, tags, etc)?

Re: Windows DNS debug logs TA not using CIM fields?

Richfez — Fri, 05 Aug 2016 14:02:08 GMT

Ryan wrote up a quick an dirty walkthrough on using Stream for DNS. You can also find a handful of things in the docs and in the splunk blogs to help. Ask if you get stuck - there are some slightly "sticky" places in the setup. BTW, you will need the winpcap libraries installed (Wireshark installs them when you install it, and they're available separately).

All the following is extracted properly - stream grabs the packets off the wire and converts them to json to send in, so all these are fields properly parsed (Splunk likes JSON... 🙂 ). The hosts and other multi-value fields are, when multi-valued, correctly done as multi-value fields. Umm, sorry, that was redundant, eh?

 { [-]
   bytes: 444
   bytes_in: 170
   bytes_out: 274
   dest_ip: 208.67.220.220
   dest_mac: 54:75:D0:BC:7E:C0
   dest_port: 53
   endtime: 2016-08-05T13:53:50.457144Z
   host_type: [ [+]
   ]
   message_type: QUERY
   name: 0108cd802801.t-1470405234.i9b462034.a3dad853f144547d6c20bd57d087602b-631-jpg.a3dad853f144547d6c20bd57d087602b-631-jpg.fp.bl.barracudabrts.com
   query: 0108cd802801.t-1470405234.i9b462034.a3dad853f144547d6c20bd57d087602b-631-jpg.a3dad853f144547d6c20bd57d087602b-631-jpg.fp.bl.barracudabrts.com
   query_type: TXT
   reply_code: NoError
   response_time: 105989
   src_ip: 10.134.0.232
   src_mac: 00:50:56:88:04:E9
   src_port: 62794
   time_taken: 105989
   timestamp: 2016-08-05T13:53:50.351155Z
   transaction_id: 54234
   transport: udp
   ttl: [ [+]
   ]
}

It really is a superior way. It may be a bit more work to get working, but then has SO much more capability making it worth it.

Re: Windows DNS debug logs TA not using CIM fields?

asieira — Fri, 05 Aug 2016 14:05:34 GMT

That's awesome, thank you for providing the additional details.

Will the JSON include all RRs in a response packet?

Re: Windows DNS debug logs TA not using CIM fields?

Richfez — Fri, 05 Aug 2016 14:07:16 GMT

Yep, the one I STARTED pasting was a bit messier, being something on Amazon with 8 responses. It was a bit messier to look at though, so I picked a simpler response for illustrative purposes. All came through as a multi-valued field and "just works".