Getting Data In

Can't get automatic extraction to work with json to kv

reswob10
Explorer

So I'm referencing this solved answer:  https://community.splunk.com/t5/Getting-Data-In/Extract-JSON-data-within-the-logs-JSON-mixed-with-un...

But my configuration isn't working.  I have this mess of a field:

Message={"ProviderGuid":"eb79061a-a566-4698-9119-3ed2807060e7","YaraMatch":[],"ProviderName":"Microsoft-Windows-DNSServer","EventName":"LOOK_UP","Opcode":0,"OpcodeName":"Info","TimeStamp":"2020-08-25T20:10:50.2211944-07:00","ThreadID":4168,"ProcessID":2632,"ProcessName":"dns","PointerSize":8,"EventDataLength":352,"XmlEventData":{"FormattedMessage":"RESPONSE_SUCCESS: TCP=0; InterfaceIP=192.168.1.5; Destination=192.168.1.50; AA=0; AD=0; QNAME=86.130.9.52.in-addr.arpa.; QTYPE=12; XID=17,307; DNSSEC=0; RCODE=0; Port=63,227; Flags=33,152; Scope=Default; Zone=..Cache; PolicyName=NULL; PacketData=439B8180000100010000000002383603...; AdditionalInfo= VirtualizationInstance:.; GUID={EC86881D-308D-4A91-94FE-5DCDDFCADFE3} ","RCODE":"0","TCP":"0","Scope":"Default","GUID":"{EC86881D-308D-4A91-94FE-5DCDDFCADFE3}","Port":"63,227","AD":"0","QNAME":"86.130.9.52.in-addr.arpa.","PolicyName":"NULL","MSec":"3243143.0166","XID":"17,307","AA":"0","Destination":"192.168.1.50","QTYPE":"12","Zone":"..Cache","PID":"2632","AdditionalInfo":"VirtualizationInstance:.","PacketData":"439B8180000100010000000002383603...","TID":"4168","ProviderName":"Microsoft-Windows-DNSServer","PName":"","DNSSEC":"0","InterfaceIP":"192.168.1.5","EventName":"LOOK_UP","Flags":"33,152"}}

and I'm trying to parse out the KV portion in the middle. 

Here are my props.conf and transforms.conf files

props.conf

[windns]
REPORT-jsonkv = report-json,report-kv


transforms.conf

[report-json]
REGEX = XmlEventData":{(?<kvdata>.+?),"

[report-kv]
REGEX = \s(\S+)=(\S+)
FORMAT = $1::$2
MV_ADD = true

 

If I understand the sequence correctly, that blob above should parse into kvdata as the following:

"FormattedMessage":"RESPONSE_SUCCESS: TCP=0; InterfaceIP=192.168.1.5; Destination=192.168.1.50; AA=0; AD=0; QNAME=86.130.9.52.in-addr.arpa.; QTYPE=12; XID=17,307; DNSSEC=0; RCODE=0; Port=63,227; Flags=33,152; Scope=Default; Zone=..Cache; PolicyName=NULL; PacketData=439B8180000100010000000002383603...; AdditionalInfo= VirtualizationInstance:.; GUID={EC86881D-308D-4A91-94FE-5DCDDFCADFE3} "

 

and then that should become kv pairs

TCP=0
InterfaceIP=192.168.1.5

and so on....

(except "AdditionalInfo" will NOT parse out due to the REGEX, but the rest should, but that's ok)

 

I have a single server, basic config.

suggestions appreciated.

0 Karma

to4kawa
Ultra Champion

what's _raw?

SEDCMD and KV_MODE=JSON are better, I guess.

0 Karma
Get Updates on the Splunk Community!

What the End of Support for Splunk Add-on Builder Means for You

Hello Splunk Community! We want to share an important update regarding the future of the Splunk Add-on Builder ...

Solve, Learn, Repeat: New Puzzle Channel Now Live

Welcome to the Splunk Puzzle PlaygroundIf you are anything like me, you love to solve problems, and what ...

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...