All Apps and Add-ons

How to modify format of MS DNS server debug log events?

Contributor

Hello,

I would like to modify format of MS DNS debug logs in order to get rid of some unimportant strings within domain names. I was playing with SEDCMD stanza in props.conf but not with success.

Log format as extracted by Splunk add-on for Microsoft DNS:
1. 2. 2017 20:19:22 0D80 PACKET 0000002548D040A0 UDP Rcv 10.17.81.32 7be7 Q [0001 D NOERROR] A (5)h42-m(3)sec(3)lab(0)

The problem is with (5)h42-m(3)sec(3)lab(0)"

I need to get events to look as follows:

2017 20:19:22 0D80 PACKET 0000002548D040A0 UDP Rcv 10.17.81.32 7be7 Q [0001 D NOERROR] A h42-m.sec.lab

When I implemented this ...
SEDCMD-removeparensnum = s/((\d))/./g
SEDCMD-removefirstperiod = s/^(.)//g
SEDCMD-removelastperiod = s/(.)$//g

... I stopped seeing my DNS logs in GUI permanently after the restart of Splunk. I do not understand. Any idea?

Tomas

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Try testing this at search time before modifying props.conf..

Try this

index=whatever | rex mode=sed s/(\(\d)\)/./g

View solution in original post

0 Karma

Path Finder

each solution had its own trouble for me.

if you use SEDCMD-remove-count = s/((\d+))/./g
you turn a PTR record into this: (.).(.).(.).(.).(.)in-addr(.)arpa(.)

also if read it correct SEDCMD happens at index time so your logs are lost if there are errors here. to have this modify the entire string like this is less than ideal.

0 Karma

Path Finder

each solution had its own trouble for me.

if you use SEDCMD-remove-count = s/((\d+))/./g
you turn a PTR record into this: (.).(.).(.).(.).(.)in-addr(.)arpa(.)

also if read it correct SEDCMD happens at index time so your logs are lost if there are errors here. to have this modify the entire string like this is less than ideal.

0 Karma

Path Finder

thanks for the fix here. this should be added standard to the app. i don't know why its logged this way and not sure why splunk app wouldn't normalize the data field.

0 Karma

Path Finder

there are a lot of issues using the SEDCMD command to try to fix this. it applies to the whole string messing up other data in the DNS log.

i found issues with all the solutions suggested here, messing up PTR records or leaving a trailing . or leading (##) in the logs.

still searching for the fix here.

0 Karma

Explorer

Did you ever find a fix you liked?

I'm curious what problems you saw in PTR records? To me, looks like they have the same weird parenthetical-count formatting problem as A records, so this fix would be common to both.

0 Karma

Motivator

Instead of recreating this by yourself, I believe the following add-on already does what you're trying to achieve: https://splunkbase.splunk.com/app/3377/

It's even CIM compliant, meaning the fields are normalized.

0 Karma

Contributor

Well, it did help but I am not really happy. I ran into three problems:

  1. Why "SEDCMD-remove1 = s/((\d))/./g" and not "SEDCMD-remove1 = s/((\d))/./g"? I am not getting logic. Seems it works the same.

  2. Once I modified props.conf with SEDCMD above, all of a sudden I am not extracting any other fields during search time (as defined in default/props.conf) - ALL OTHER FIELDS VANISHED. I am getting just host, source, sourcetype.

  3. adding "SEDCMD-remove-head-dot = s/\s(.)//g" into props.conf does not do anything (was working with rex in search bar)

  4. adding "SEDCMD-remove-tail-dot = s/(.)$//g" into props.conf does not do anything (was warking with rex in search bar)

I do not understand Splunk's logic. Simply not.

0 Karma

Contributor

I can confirm now that skoelpin's solution works!

My props.conf

[MSAD:NT6:DNS]

Replace (3)www(6)google(3)com with www.google.com etc.

SEDCMD-remove-count = s/((\d+))/./g
SEDCMD-remove-head-dot = s/\s(.)//g
SEDCMD-remove-tail-dot = s/(.)$//g

0 Karma

SplunkTrust
SplunkTrust

Try testing this at search time before modifying props.conf..

Try this

index=whatever | rex mode=sed s/(\(\d)\)/./g

View solution in original post

0 Karma

Contributor

Yes, it does work correctly in this first stage.

  1. 2. 2017 23:07:08 0D7C PACKET 0000002549C0C0A0 UDP Snd 10.18.1.51 b1aa Q [0000 NOERROR] SRV .ldap.tcp(23)Default-First-Site-Name.sites.dc.msdcs.develop3.develop2.develop.local.
0 Karma

SplunkTrust
SplunkTrust

So you verified its working correctly at search time.. Do you want me to give you the SEDCMD so you can add it to your props.conf for index time now?

0 Karma

Contributor

Yes. Please.

0 Karma

SplunkTrust
SplunkTrust

Place this in your props.conf under $SPLUNK_HOME\etc\apps\#APP_NAME\local

[YourSourcetype]
SEDCMD-remove_parens = s/(\(\d)\)/./g

Don't forget to restart the Splunk service after making this change.

Lastly, if this works for you then please accept the answer

0 Karma