Splunk Search

How to extract dynamic key value pairs in mixed data from DB Connect app?

TangentTexan
New Member

I am indexing a string for the DBConnect where one of the fields stores a modified data in one of the cells.
In a sub part of a Log entry I am having a difficulty extracting the KV pairs. Where the previous parts of each log entry have the basic Key=Value on a line each, that is working. The part that is not is from the AllXML field - string= item.

  1. Starts with:
    string="
  2. Each key:value pair is separated with:
    :
  3. Additional Keys begin from:
    

  4. Additional Values end with:
    
  5. Stops with:
    "

Sample Log Entry:

AllXML=<error
    application="my application name"
    detail="whole bunch of stuff"
    time="2014-08-08T11:11:59.4225842Z"
    statusCode="500">   
<serverVariables>
        <item
          name="ALL_HTTP">
          <value    
    string="KEYNAME1:Value1 Value1,MoreValue1&#xD;&#xA;Key-Name2:Value2-Value2.Value2&#xD;&#xA;KEY-NAME3:true&#xD;&#xA;Key_Name4:Value4 ; Value4,Value4.Value4 - vvvvvvAAALLLLuuuueeeee44444&#xD;&#xA;" />

The OutPut should be, so that these items get indexed:

application="my application name"
detail="whole bunch of stuff
time=2014-08-08T11:11:59.4225842Z
statusCode=500
KEYNAME1=Value1
Key-Name2=Value2-Value2.Value2
KEY-NAME3=true
Key_Name4=Value4 ; Value4,Value4.Value4 - vvvvvvAAALLLLuuuueeeee44444

I know it should be through the transform.conf - but getting the regex to pull the data in has become difficult.
ant assistance would be appreciated.
Thanks,

0 Karma

somesoni2
Revered Legend

Try this

In props.conf

[YourSourceType]
REPORT-customkvtransform = customkvextract

In transforms.conf

[customkvextract]
CLEAN_KEYS = 1
FORMAT = $2::$3
MV_ADD = 0
REGEX = ([&#xA;]*)([^:]+):([^&]+)&#xD;
SOURCE_KEY = string
0 Karma
Get Updates on the Splunk Community!

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...

Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0

Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...