topic Re: How to configure heavy forwarder to extract multivalue nested json ? in Getting Data In

How to configure heavy forwarder to extract multivalue nested json ?

prashant5847 — Thu, 29 Jun 2023 18:09:25 GMT

I have following set up in place and I am sending events to splunk cloud from K8S cluster. I am using HF for data manipulation.

K8S cluster --> Heavy Forwarder --> Splunk Cloud

I received all events send by k8s cluster but not all field from events are getting extracted in json.

Current Output


{ [-]
  action: modify
  containerid: 278e7bddd8b50ad885077
  count: 1
  host: example.com
  pid: 125
  time: 1456789023
  timestamp: 14567890234356
  metrics: {"metrics":{\"name1\":{\"m1\":\"downsample\",\"m2\":\"sum\"},\"name2\":{\"Headers\":{\"Selector\":{\"m1\":\"downsample\",\"m2\":\"sum\"}}}"}
  uid: 0
  user: 0
}

Looking for convert all data in JSON key value format as shown in expected output.

Expected Output

{ [-] action: modify containerid: 278e7bddd8b50ad885077 count: 1 host: example.com pid: 125 time: 1456789023 timestamp: 14567890234356 metrics: {[-] metrics:{ [-] name1:{ [-] m1: downsample m2: sum } name2:{ [-] Headers :{ [-] Selector :{ [-] m1: downsample m2: sum } } } } } uid: 0 user: 0 }

How I need to configure Splunk Heavy Forwarder to extract multivalued nested json ?

Re: How to configure heavy forwarder to extract multivalue nested json ?

venkatasri — Mon, 03 Jul 2023 23:00:31 GMT

Hi @prashant5847 , Splunk is good at auto extracting the JSON event if it is well formatted.

Did you search it on Search head and saw the fields not being extracted correctly? Choose search mode - Smart/Verbose.

If you still can not find it, you don't need to extract it on Heavy forwarder. Search head will do the extractions for you.
Either you could use spath command - Example <your search return json event> | spath - This is inline search.

Or update source type settings (props.conf) on search head to include following config-

[<sourcetype-name-of-json-events>]
AUTO_KV_JSON = true
KV_MODE = json

Hope this helps!

------------------------------

Srikanth Yarlagadda.

Re: How to configure heavy forwarder to extract multivalue nested json ?

prashant5847 — Wed, 12 Jul 2023 08:46:32 GMT

Hi @venkatasri

Thank you for your suggestion. Yes, after formatting data in proper JSON format, it extracted successfully.

Further, we would like to move 'm1: downsample' key value pair which is in Selector in same JSON message header part below host and above pid. Is this possible from splunk heavy forwarder ? If yes, what configuration changes I need to apply on the message ?

Thank you,
Prashant