How to configure heavy forwarder to extract multivalue nested json ?

[prashant5847]

I have following set up in place and I am sending events to splunk cloud from K8S cluster. I am using HF for data manipulation. 

K8S cluster  --> Heavy Forwarder --> Splunk Cloud

I received all events send by k8s cluster but not all field from events are getting extracted in json. 

Current Output

{ [-]
  action: modify
  containerid: 278e7bddd8b50ad885077
  count: 1
  host: example.com
  pid: 125
  time: 1456789023
  timestamp: 14567890234356
  metrics: {"metrics":{\"name1\":{\"m1\":\"downsample\",\"m2\":\"sum\"},\"name2\":{\"Headers\":{\"Selector\":{\"m1\":\"downsample\",\"m2\":\"sum\"}}}"}
  uid: 0
  user: 0
}

Looking for convert all data in JSON key value format as shown in expected output. 

Expected Output

{ [-]
  action: modify
  containerid: 278e7bddd8b50ad885077
  count: 1
  host: example.com
  pid: 125
  time: 1456789023
  timestamp: 14567890234356
  metrics: {[-]
     metrics:{ [-]
         name1:{ [-]
           m1: downsample
           m2: sum
         }
         name2:{ [-]
           Headers :{ [-]
             Selector :{ [-]
               m1: downsample
               m2: sum
             }
           }
         }
     }
  }
  uid: 0
  user: 0
}

  How I need to configure Splunk Heavy Forwarder to extract  multivalued nested json ?

[prashant5847]

Hi @venkatasri 

Thank you for your suggestion. Yes, after formatting data in proper JSON format, it extracted successfully. 

Further, we would like to move 'm1:  downsample' key value pair which is in Selector in same JSON message header part below host and above pid. Is this possible from splunk heavy forwarder ? If yes, what configuration changes I need to apply on the message ?

Thank you,
Prashant

[venkatasri]

Hi @prashant5847 , Splunk is good at auto extracting the JSON event if it is well formatted.

Did you search it on Search head and saw the fields not being extracted correctly? Choose search mode - Smart/Verbose.

If you still can not find it, you don't need to extract it on Heavy forwarder. Search head will do the extractions for you.
Either you could use spath command - Example  <your search return json event> | spath  - This is inline search.

Or update source type settings (props.conf) on search head to include following config-

[<sourcetype-name-of-json-events>]
AUTO_KV_JSON = true
KV_MODE = json

Hope this helps!

------------------------------

Srikanth Yarlagadda.