How do I index only certain elements from JSON in splunk with Custom Handler

[mlcooper7]

I am successfully pulling a JSON array with the REST API Modular input and it is indexing each item in the array as separate events in Splunk which is exactly what I want.

My handler is here:

class foo2:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
        if response_type == "json":
            output = json.loads(raw_response_output)
            for element in output["result"]:
                print_xml_stream(json.dumps(element))

        else:
            print_xml_stream(raw_response_output)

However, the JSON payload is large and I only need some of the values from each of the array elements to be indexed into splunk and the rest can be discarded.

Here is an example of my JSON which could have 100 items in the result array:

{
  "result": [
    {
      "calendar_integration": "1",
      "country": "dd3",
      "last_position_update": "val1",
      "u_hr_profile": {
        "link": "foo.biz",
        "value": "8c00077d4f0a5e04ab19defd0210c757"
      },
      "user_password": "fake",
      "last_login_time": "",
      "u_cms_hr_language": "en",
      "x_pd_integration_pagerduty_id": "",
      "u_reporting_relationships": "",
      "source": "",
      "sys_updated_on": "2016-07-29 06:20:36",
      "building": "skyscraper",
      "u_sap_segment": {
        "link": "https://foo.bar",
        "value": "ddde2c7237291e007e5e27d2b3990eb9"
      }
    },
    {
      "calendar_integration": "1",
      "country": "dd3",
      "last_position_update": "",
      "u_hr_profile": {
        "link": "foo.biz",
        "value": "8c00077d4f0a5e04ab19defd0210c757"
      },
      "user_password": "",
      "last_login_time": "",
      "u_cms_hr_language": "",
      "x_pd_integration_pagerduty_id": "",
      "u_reporting_relationships": "",
      "source": "",
      "sys_updated_on": "2016-07-29 06:20:36",
      "building": "tower",
      "u_sap_segment": {
        "link": "https://foo.com.us",
        "value": "ddde2c7237291e007e5e27d2b3990eb9"
      }
    }
  ]
}

I don't need every key:value pair in each element of the array.

How can I modify my handler script to only pull out country, u_hr_profile and building and index only those 3 key:value pairs from each element of the array into Splunk?

[Damien_Dallimor · Accepted solution]

class foo2:

     def __init__(self,**args):
         pass

     def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
         if response_type == "json":
             output = json.loads(raw_response_output)
             for element in output["result"]:
                 output_json = {}
                 output_json['country'] = element['country']
                 output_json['u_hr_profile'] = element['u_hr_profile']
                 output_json['building'] = element['building']
                 print_xml_stream(json.dumps(output_json))

         else:
             print_xml_stream(raw_response_output)

[mlcooper7]

Thank you very much, perfect!