Splunk Search

Regex help for Bind DNS logs

jwalzerpitt
Influencer

At some point in the past month, the existing extract in transforms.conf quit working and the DNS logs (ingesting from BlueCat syslog), specifically queries, are no longer being parsed correctly. The extract in transforms.conf I have is:

EXTRACT-process,pid,src,src_port,query = \d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\S+\s\S+\s(?P<process>\S+)\[(?<pid>\d+)\]\:\s+\S+\s+(?P<src>\S+)\#(?P<src_port>\S+)(\/\s|\s)\((?P<query>\S+)\)

Nothing has changed in the custom TA and as far as I can see the format of the DNS logs has not changed before/after parsing stopped working

With that, I started to create a regex to replace the extract in transforms.conf. The regex is as follows:

client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\s(?<message_type>query):\s(?<query>\S+)\s(?<dns_request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*)\s\((?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})\)

I tested it against generic query events with success:

2019-07-30 08:59:20 8.8.8.8 BLUECAT-LOG-MSG named[311]: client @0x7fc2c5f35e30 10.1.1.1#57195 (e1875.dscg.akamaiedge.net): view default: query: e1875.dscg.akamaiedge.net IN AAAA + (8.8.8.8)

While reviewing DNS query logs, I see a different query related event for cached queries:

2019-07-30 09:50:01 8.8.8.8 BLUECAT-LOG-MSG named[1054]: client @0x7f9f9bff1230 10.1.1.1#35281 (acp-ss-ue1.adobe.io): view default: query (cache) 'acp-ss-ue1.adobe.io/A/IN' denied

How can I modify my regex to include cached query events, or am I better off creating a separate query for them?

Thx

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Try this regex:

client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<dns_request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*)\s\((?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})\)| \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<dns_request_class_name>[^']+))
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this regex:

client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<dns_request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*)\s\((?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})\)| \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<dns_request_class_name>[^']+))
---
If this reply helps you, Karma would be appreciated.
0 Karma

jwalzerpitt
Influencer

Rich,

Thx for the reply.

Plugging in the regex you listed:

 (?J)client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<dns_request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*)\s\((?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})\)| \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<dns_request_class_name>[^']+))

Returns the following error from the point of '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<dns_request_class_name>[^']+))

(? A subpattern name must be unique
) A subpattern name must be unique
(? A subpattern name must be unique
) A subpattern name must be unique
(? A subpattern name must be unique
) A subpattern name must be unique

Thx

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Sorry about that. I left out the (?J) flag. Corrected.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jwalzerpitt
Influencer

NP at all - added the (?J) as a global modifier at https://regex101.com/ and was able to match on both events - regular query and cache query.

I modified the regex to remove 'host' as I am already pulling that via transforms.conf.

Updated regex is:

client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*) | \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<request_class_name>[^']+))

Via props.conf and transforms.conf I am pulling the host as follows:
[props.conf]

TRANSFORMS-host = bluecat_dns-host
FIELDALIAS-dns = host AS dns

[transforms.conf]

[bluecat_dns-host]
DEST_KEY = MetaData:Host
REGEX = \d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s(\S+)\s 
FORMAT = host::$1

With all of that, can I replace the extract in props.conf:

EXTRACT-process,pid,src,src_port,query = \d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\S+\s\S+\s(?P<process>\S+)\[(?<pid>\d+)\]\:\s+\S+\s+(?P<src>\S+)\#(?P<src_port>\S+)(\/\s|\s)\((?P<query>\S+)\)

With the new extract:

EXTRACT-src,src_port,message_type,query,request_class_name,record_type,flag,host = (?J) client\s@.+\s(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|(?:::)?(?:[a-zA-Z\d]{1,4}::?){1,7}[a-zA-Z\d]{0,4})#(?<src_port>\d+).*\sdefault:\s(?<message_type>query)(: (?<query>\S+) (?<request_class_name>\w+)\s(?<record_type>\w+)\s(?<flag>(?:\+|\-)\S*) | \(cache\) '(?<query>[^\/]+)\/(?<record_type>[^\/]+)\/(?<request_class_name>[^']+))

Thx

0 Karma

richgalloway
SplunkTrust
SplunkTrust

That should work.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jwalzerpitt
Influencer

TYVM Rich!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...