All Apps and Add-ons
Highlighted

How do I index only certain elements from JSON in splunk with Custom Handler

New Member

I am successfully pulling a JSON array with the REST API Modular input and it is indexing each item in the array as separate events in Splunk which is exactly what I want.

My handler is here:

class foo2:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
        if response_type == "json":
            output = json.loads(raw_response_output)
            for element in output["result"]:
                print_xml_stream(json.dumps(element))

        else:
            print_xml_stream(raw_response_output)

However, the JSON payload is large and I only need some of the values from each of the array elements to be indexed into splunk and the rest can be discarded.

Here is an example of my JSON which could have 100 items in the result array:

{
  "result": [
    {
      "calendar_integration": "1",
      "country": "dd3",
      "last_position_update": "val1",
      "u_hr_profile": {
        "link": "foo.biz",
        "value": "8c00077d4f0a5e04ab19defd0210c757"
      },
      "user_password": "fake",
      "last_login_time": "",
      "u_cms_hr_language": "en",
      "x_pd_integration_pagerduty_id": "",
      "u_reporting_relationships": "",
      "source": "",
      "sys_updated_on": "2016-07-29 06:20:36",
      "building": "skyscraper",
      "u_sap_segment": {
        "link": "https://foo.bar",
        "value": "ddde2c7237291e007e5e27d2b3990eb9"
      }
    },
    {
      "calendar_integration": "1",
      "country": "dd3",
      "last_position_update": "",
      "u_hr_profile": {
        "link": "foo.biz",
        "value": "8c00077d4f0a5e04ab19defd0210c757"
      },
      "user_password": "",
      "last_login_time": "",
      "u_cms_hr_language": "",
      "x_pd_integration_pagerduty_id": "",
      "u_reporting_relationships": "",
      "source": "",
      "sys_updated_on": "2016-07-29 06:20:36",
      "building": "tower",
      "u_sap_segment": {
        "link": "https://foo.com.us",
        "value": "ddde2c7237291e007e5e27d2b3990eb9"
      }
    }
  ]
}

I don't need every key:value pair in each element of the array.

How can I modify my handler script to only pull out country, u_hr_profile and building and index only those 3 key:value pairs from each element of the array into Splunk?

0 Karma
Highlighted

Re: How do I index only certain elements from JSON in splunk with Custom Handler

Ultra Champion
class foo2:

     def __init__(self,**args):
         pass

     def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
         if response_type == "json":
             output = json.loads(raw_response_output)
             for element in output["result"]:
                 output_json = {}
                 output_json['country'] = element['country']
                 output_json['u_hr_profile'] = element['u_hr_profile']
                 output_json['building'] = element['building']
                 print_xml_stream(json.dumps(output_json))

         else:
             print_xml_stream(raw_response_output)

View solution in original post

Highlighted

Re: How do I index only certain elements from JSON in splunk with Custom Handler

New Member

Thank you very much, perfect!

0 Karma