topic Re: Best way to nullQueue DNS logs by source in Getting Data In

Best way to nullQueue DNS logs by source

adalbor — Wed, 11 Mar 2020 15:33:51 GMT

Hey All,

Was just curious if there was a more efficient way of dropping DNS events by the actual query source rather than what I have below.

[MSAD:NT6:DNS]
TRANSFORMS-dropdns=dropdns

[dropdns]
REGEX=.*IPOFSOURCE.*
DEST_KEY=queue
FORMAT=nullQueue

adalbor — Mon, 16 Mar 2020 17:37:54 GMT

I modified it to include another IP but it doesnt appear to be working. Have this on all of my IDX's.
Any suggestions?

[MSAD:NT6:DNS]
 TRANSFORMS-dropdns = dropdns

 [dropdns]
 REGEX=.*1.1.1.1.*|.*2.2.2.2.*
 DEST_KEY=queue
 FORMAT=nullQueue

anmolpatel — Mon, 16 Mar 2020 23:55:40 GMT

@adalbor need two pieces of info:
1) what is the deployment architecture?
2) Did you reload the indexers to get the new props and transforms config ?

adalbor — Tue, 17 Mar 2020 13:37:28 GMT

1) Clustered indexers and UF's on the servers with the DNS logs
2) Yes after every change

adalbor — Tue, 24 Mar 2020 15:47:22 GMT

I ended up resolving the issue I had. The UF in question that wasn't working was going through a HF rather than straight to my IDX's

Applied the following to the HF's and IDX's and it started dropping the matching events.

Support also recommended I use source rather than sourcetype as it was more reliable.

Props.conf
[source::c:\DNSLOGS\dns.log]
TRANSFORMS-dropdns=dropdns1,dropdns2

Transforms.conf
[dropdns1]
REGEX = .1.1.1.1.
DEST_KEY = queue
FORMAT = nullQueue

[dropdns2]
REGEX = .2.2.2.2.
DEST_KEY = queue
FORMAT = nullQueue