Getting Data In

Can't get automatic extraction to work with json to kv

reswob10
Explorer

So I'm referencing this solved answer:  https://community.splunk.com/t5/Getting-Data-In/Extract-JSON-data-within-the-logs-JSON-mixed-with-un...

But my configuration isn't working.  I have this mess of a field:

Message={"ProviderGuid":"eb79061a-a566-4698-9119-3ed2807060e7","YaraMatch":[],"ProviderName":"Microsoft-Windows-DNSServer","EventName":"LOOK_UP","Opcode":0,"OpcodeName":"Info","TimeStamp":"2020-08-25T20:10:50.2211944-07:00","ThreadID":4168,"ProcessID":2632,"ProcessName":"dns","PointerSize":8,"EventDataLength":352,"XmlEventData":{"FormattedMessage":"RESPONSE_SUCCESS: TCP=0; InterfaceIP=192.168.1.5; Destination=192.168.1.50; AA=0; AD=0; QNAME=86.130.9.52.in-addr.arpa.; QTYPE=12; XID=17,307; DNSSEC=0; RCODE=0; Port=63,227; Flags=33,152; Scope=Default; Zone=..Cache; PolicyName=NULL; PacketData=439B8180000100010000000002383603...; AdditionalInfo= VirtualizationInstance:.; GUID={EC86881D-308D-4A91-94FE-5DCDDFCADFE3} ","RCODE":"0","TCP":"0","Scope":"Default","GUID":"{EC86881D-308D-4A91-94FE-5DCDDFCADFE3}","Port":"63,227","AD":"0","QNAME":"86.130.9.52.in-addr.arpa.","PolicyName":"NULL","MSec":"3243143.0166","XID":"17,307","AA":"0","Destination":"192.168.1.50","QTYPE":"12","Zone":"..Cache","PID":"2632","AdditionalInfo":"VirtualizationInstance:.","PacketData":"439B8180000100010000000002383603...","TID":"4168","ProviderName":"Microsoft-Windows-DNSServer","PName":"","DNSSEC":"0","InterfaceIP":"192.168.1.5","EventName":"LOOK_UP","Flags":"33,152"}}

and I'm trying to parse out the KV portion in the middle. 

Here are my props.conf and transforms.conf files

props.conf

[windns]
REPORT-jsonkv = report-json,report-kv


transforms.conf

[report-json]
REGEX = XmlEventData":{(?<kvdata>.+?),"

[report-kv]
REGEX = \s(\S+)=(\S+)
FORMAT = $1::$2
MV_ADD = true

 

If I understand the sequence correctly, that blob above should parse into kvdata as the following:

"FormattedMessage":"RESPONSE_SUCCESS: TCP=0; InterfaceIP=192.168.1.5; Destination=192.168.1.50; AA=0; AD=0; QNAME=86.130.9.52.in-addr.arpa.; QTYPE=12; XID=17,307; DNSSEC=0; RCODE=0; Port=63,227; Flags=33,152; Scope=Default; Zone=..Cache; PolicyName=NULL; PacketData=439B8180000100010000000002383603...; AdditionalInfo= VirtualizationInstance:.; GUID={EC86881D-308D-4A91-94FE-5DCDDFCADFE3} "

 

and then that should become kv pairs

TCP=0
InterfaceIP=192.168.1.5

and so on....

(except "AdditionalInfo" will NOT parse out due to the REGEX, but the rest should, but that's ok)

 

I have a single server, basic config.

suggestions appreciated.

0 Karma

to4kawa
Ultra Champion

what's _raw?

SEDCMD and KV_MODE=JSON are better, I guess.

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...