Can't get automatic extraction to work with json to kv

reswob10 — Wed, 26 Aug 2020 03:27:48 GMT

So I'm referencing this solved answer: https://community.splunk.com/t5/Getting-Data-In/Extract-JSON-data-within-the-logs-JSON-mixed-with-unstructured/td-p/195292/page/2

But my configuration isn't working. I have this mess of a field:

Message={"ProviderGuid":"eb79061a-a566-4698-9119-3ed2807060e7","YaraMatch":[],"ProviderName":"Microsoft-Windows-DNSServer","EventName":"LOOK_UP","Opcode":0,"OpcodeName":"Info","TimeStamp":"2020-08-25T20:10:50.2211944-07:00","ThreadID":4168,"ProcessID":2632,"ProcessName":"dns","PointerSize":8,"EventDataLength":352,"XmlEventData":{"FormattedMessage":"RESPONSE_SUCCESS: TCP=0; InterfaceIP=192.168.1.5; Destination=192.168.1.50; AA=0; AD=0; QNAME=86.130.9.52.in-addr.arpa.; QTYPE=12; XID=17,307; DNSSEC=0; RCODE=0; Port=63,227; Flags=33,152; Scope=Default; Zone=..Cache; PolicyName=NULL; PacketData=439B8180000100010000000002383603...; AdditionalInfo= VirtualizationInstance:.; GUID={EC86881D-308D-4A91-94FE-5DCDDFCADFE3} ","RCODE":"0","TCP":"0","Scope":"Default","GUID":"{EC86881D-308D-4A91-94FE-5DCDDFCADFE3}","Port":"63,227","AD":"0","QNAME":"86.130.9.52.in-addr.arpa.","PolicyName":"NULL","MSec":"3243143.0166","XID":"17,307","AA":"0","Destination":"192.168.1.50","QTYPE":"12","Zone":"..Cache","PID":"2632","AdditionalInfo":"VirtualizationInstance:.","PacketData":"439B8180000100010000000002383603...","TID":"4168","ProviderName":"Microsoft-Windows-DNSServer","PName":"","DNSSEC":"0","InterfaceIP":"192.168.1.5","EventName":"LOOK_UP","Flags":"33,152"}}

and I'm trying to parse out the KV portion in the middle.

Here are my props.conf and transforms.conf files

props.conf

[windns]
REPORT-jsonkv = report-json,report-kv

transforms.conf

[report-json]
REGEX = XmlEventData":{(?<kvdata>.+?),"

[report-kv]
REGEX = \s(\S+)=(\S+)
FORMAT = $1::$2
MV_ADD = true

If I understand the sequence correctly, that blob above should parse into kvdata as the following:

"FormattedMessage":"RESPONSE_SUCCESS: TCP=0; InterfaceIP=192.168.1.5; Destination=192.168.1.50; AA=0; AD=0; QNAME=86.130.9.52.in-addr.arpa.; QTYPE=12; XID=17,307; DNSSEC=0; RCODE=0; Port=63,227; Flags=33,152; Scope=Default; Zone=..Cache; PolicyName=NULL; PacketData=439B8180000100010000000002383603...; AdditionalInfo= VirtualizationInstance:.; GUID={EC86881D-308D-4A91-94FE-5DCDDFCADFE3} "

and then that should become kv pairs

TCP=0
InterfaceIP=192.168.1.5

and so on....

(except "AdditionalInfo" will NOT parse out due to the REGEX, but the rest should, but that's ok)

I have a single server, basic config.

suggestions appreciated.

Re: Can't get automatic extraction to work with json to kv

to4kawa — Thu, 27 Aug 2020 00:29:08 GMT

what's _raw?

SEDCMD and KV_MODE=JSON are better, I guess.

topic Can't get automatic extraction to work with json to kv in Getting Data In

Can't get automatic extraction to work with json to kv

Re: Can't get automatic extraction to work with json to kv