Splunk Search

Regex help for Bind DNS logs

Motivator

At some point in the past month, the existing extract in transforms.conf quit working and the DNS logs (ingesting from BlueCat syslog), specifically queries, are no longer being parsed correctly. The extract in transforms.conf I have is:

EXTRACT-process,pid,src,src_port,query = \d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\S+\s\S+\s(?P<process>\S+)\[(?<pid>\d+)\]\:\s+\S+\s+(?P<src>\S+)\#(?P<src_port>\S+)(\/\s|\s)\((?P<query>\S+)\)

Nothing has changed in the custom TA and as far as I can see the format of the DNS logs has not changed before/after parsing stopped working

With that, I started to create a regex to replace the extract in transforms.conf. The regex is as follows:

client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\s(?<message_type>query):\s(?<query>\S+)\s(?<dns_request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*)\s\((?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})\)

I tested it against generic query events with success:

2019-07-30 08:59:20 8.8.8.8 BLUECAT-LOG-MSG named[311]: client @0x7fc2c5f35e30 10.1.1.1#57195 (e1875.dscg.akamaiedge.net): view default: query: e1875.dscg.akamaiedge.net IN AAAA + (8.8.8.8)

While reviewing DNS query logs, I see a different query related event for cached queries:

2019-07-30 09:50:01 8.8.8.8 BLUECAT-LOG-MSG named[1054]: client @0x7f9f9bff1230 10.1.1.1#35281 (acp-ss-ue1.adobe.io): view default: query (cache) 'acp-ss-ue1.adobe.io/A/IN' denied

How can I modify my regex to include cached query events, or am I better off creating a separate query for them?

Thx

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Try this regex:

client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<dns_request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*)\s\((?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})\)| \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<dns_request_class_name>[^']+))
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Try this regex:

client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<dns_request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*)\s\((?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})\)| \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<dns_request_class_name>[^']+))
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

0 Karma

Motivator

Rich,

Thx for the reply.

Plugging in the regex you listed:

 (?J)client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<dns_request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*)\s\((?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})\)| \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<dns_request_class_name>[^']+))

Returns the following error from the point of '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<dns_request_class_name>[^']+))

(? A subpattern name must be unique
) A subpattern name must be unique
(? A subpattern name must be unique
) A subpattern name must be unique
(? A subpattern name must be unique
) A subpattern name must be unique

Thx

0 Karma

SplunkTrust
SplunkTrust

Sorry about that. I left out the (?J) flag. Corrected.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Motivator

NP at all - added the (?J) as a global modifier at https://regex101.com/ and was able to match on both events - regular query and cache query.

I modified the regex to remove 'host' as I am already pulling that via transforms.conf.

Updated regex is:

client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*) | \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<request_class_name>[^']+))

Via props.conf and transforms.conf I am pulling the host as follows:
[props.conf]

TRANSFORMS-host = bluecat_dns-host
FIELDALIAS-dns = host AS dns

[transforms.conf]

[bluecat_dns-host]
DEST_KEY = MetaData:Host
REGEX = \d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s(\S+)\s 
FORMAT = host::$1

With all of that, can I replace the extract in props.conf:

EXTRACT-process,pid,src,src_port,query = \d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\S+\s\S+\s(?P<process>\S+)\[(?<pid>\d+)\]\:\s+\S+\s+(?P<src>\S+)\#(?P<src_port>\S+)(\/\s|\s)\((?P<query>\S+)\)

With the new extract:

EXTRACT-src,src_port,message_type,query,request_class_name,record_type,flag,host = (?J) client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*) | \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<request_class_name>[^']+))

Thx

0 Karma

SplunkTrust
SplunkTrust

That should work.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Motivator

TYVM Rich!

0 Karma