All Apps and Add-ons

How can I limit my collection of DNS data using STREAM to a single domain name.

edhealea
Path Finder

I currently have stream collecting DNS from our DNS server. I also have some DNS forwarder that I have been requested to capture any query or responses to a particular DNS name.
I though I could just turn on Stream on the DNS forwarders and write a props/transforms.conf and apply it to those UFs and it would work but it didn't. I get everything from the DNS forwarders.
This is the props and transforms that I used.
props.conf
[stream:dns]
TRANSFORMS-set= setnull, setparsing

transforms.conf
[setnull]
REGEX=^.*
DEST_KEY=queue
FORMAT=nullQueue

Stream config - Remove all data coming in

[setparsing]
REGEX=^.?(?:query|response)\":[\"(?i)(?:dns.name.here).*$
DEST_KEY=queue
FORMAT=indexQueue

Stream config - only collect data for listed url

0 Karma
1 Solution

edhealea
Path Finder

This is what I have found.
You need to call the host from within the prop.conf file not within the tansforms.conf. I couldn't find why this is true on an example @
https://answers.splunk.com/answers/75881/filtering-on-host.html

This is what I end up with and it is working as expected.

Props.conf
[stream:dns]
TRANSFORMS-null= setnull_stream_dns
[host::(servernameAA\d*)
TRANSFORMS-DMZnull= setnull_dmzdnsfwd, dmzdnsfwd_setparsing

Transforms.conf
[setnull_dmzdnsfwd]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

Stream config - Remove all DNZ DNS Forwarders

[dmzdnsfwd_setparsing]
REGEX = ^.?(?:query|response)\":[\"(?i)(?:dns.name.here).$
DEST_KEY = queue
FORMAT = indexQueue

Stream config - only collect data for listed url

[setnull_stream_dns]
REGEX = ^.?(?:query|response)\":[\"(?i)(?:^empty|.?live.com|.?microsoft.com|.?office365.com).*$
DEST_KEY = queue
FORMAT = nullQueue

Stream config

View solution in original post

0 Karma

edhealea
Path Finder

This is what I have found.
You need to call the host from within the prop.conf file not within the tansforms.conf. I couldn't find why this is true on an example @
https://answers.splunk.com/answers/75881/filtering-on-host.html

This is what I end up with and it is working as expected.

Props.conf
[stream:dns]
TRANSFORMS-null= setnull_stream_dns
[host::(servernameAA\d*)
TRANSFORMS-DMZnull= setnull_dmzdnsfwd, dmzdnsfwd_setparsing

Transforms.conf
[setnull_dmzdnsfwd]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

Stream config - Remove all DNZ DNS Forwarders

[dmzdnsfwd_setparsing]
REGEX = ^.?(?:query|response)\":[\"(?i)(?:dns.name.here).$
DEST_KEY = queue
FORMAT = indexQueue

Stream config - only collect data for listed url

[setnull_stream_dns]
REGEX = ^.?(?:query|response)\":[\"(?i)(?:^empty|.?live.com|.?microsoft.com|.?office365.com).*$
DEST_KEY = queue
FORMAT = nullQueue

Stream config

0 Karma

edhealea
Path Finder

Since the UFs don't parse data, I moved my P&T back to my HFs handling this chore. I also changed the P&T by merging it with my current TA_stream P&T. Now that I have all the my DNS Streams coming to my HFs, I have to be able to let the DNS servers send everything through but limit my DNS forwarders to only allow the single Domain queries/responses through so this is what I came up with which still doesn't work. I get everything from both the DNS servers and the DNS forwarders.
props.conf
[stream:dns]
TRANSFORMS-null=setnull_stream_dns, setparsing, setnull_fwd

transforms.conf
[setnull_fwd]
SOURCE_KEY = MetaData:Host
REGEX = ^DNS_FWD*
DEST_KEY = queue
FORMAT = nullQueue
# Stream config - Remove all DNZ DNS Forwarders

[setparsing]
REGEX=^.?(?:query|response)\":[\"(?i)(?:dns.name.here).$
DEST_KEY=queue
FORMAT=indexQueue

Stream config - only collect data for listed url

[setnull_stream_dns]
REGEX=^.?(?:query|response)\":[\"(?i)(?:^empty|.?live.com|.?microsoft.com|.?office365.com).*$
DEST_KEY=queue
FORMAT=nullQueue

Stream config

0 Karma
Get Updates on the Splunk Community!

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...

Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0

Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...