All Apps and Add-ons

How do I index only certain elements from JSON in splunk with Custom Handler

mlcooper7
New Member

I am successfully pulling a JSON array with the REST API Modular input and it is indexing each item in the array as separate events in Splunk which is exactly what I want.

My handler is here:

class foo2:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
        if response_type == "json":
            output = json.loads(raw_response_output)
            for element in output["result"]:
                print_xml_stream(json.dumps(element))

        else:
            print_xml_stream(raw_response_output)

However, the JSON payload is large and I only need some of the values from each of the array elements to be indexed into splunk and the rest can be discarded.

Here is an example of my JSON which could have 100 items in the result array:

{
  "result": [
    {
      "calendar_integration": "1",
      "country": "dd3",
      "last_position_update": "val1",
      "u_hr_profile": {
        "link": "foo.biz",
        "value": "8c00077d4f0a5e04ab19defd0210c757"
      },
      "user_password": "fake",
      "last_login_time": "",
      "u_cms_hr_language": "en",
      "x_pd_integration_pagerduty_id": "",
      "u_reporting_relationships": "",
      "source": "",
      "sys_updated_on": "2016-07-29 06:20:36",
      "building": "skyscraper",
      "u_sap_segment": {
        "link": "https://foo.bar",
        "value": "ddde2c7237291e007e5e27d2b3990eb9"
      }
    },
    {
      "calendar_integration": "1",
      "country": "dd3",
      "last_position_update": "",
      "u_hr_profile": {
        "link": "foo.biz",
        "value": "8c00077d4f0a5e04ab19defd0210c757"
      },
      "user_password": "",
      "last_login_time": "",
      "u_cms_hr_language": "",
      "x_pd_integration_pagerduty_id": "",
      "u_reporting_relationships": "",
      "source": "",
      "sys_updated_on": "2016-07-29 06:20:36",
      "building": "tower",
      "u_sap_segment": {
        "link": "https://foo.com.us",
        "value": "ddde2c7237291e007e5e27d2b3990eb9"
      }
    }
  ]
}

I don't need every key:value pair in each element of the array.

How can I modify my handler script to only pull out country, u_hr_profile and building and index only those 3 key:value pairs from each element of the array into Splunk?

0 Karma
1 Solution

Damien_Dallimor
Ultra Champion
class foo2:

     def __init__(self,**args):
         pass

     def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
         if response_type == "json":
             output = json.loads(raw_response_output)
             for element in output["result"]:
                 output_json = {}
                 output_json['country'] = element['country']
                 output_json['u_hr_profile'] = element['u_hr_profile']
                 output_json['building'] = element['building']
                 print_xml_stream(json.dumps(output_json))

         else:
             print_xml_stream(raw_response_output)

View solution in original post

Damien_Dallimor
Ultra Champion
class foo2:

     def __init__(self,**args):
         pass

     def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
         if response_type == "json":
             output = json.loads(raw_response_output)
             for element in output["result"]:
                 output_json = {}
                 output_json['country'] = element['country']
                 output_json['u_hr_profile'] = element['u_hr_profile']
                 output_json['building'] = element['building']
                 print_xml_stream(json.dumps(output_json))

         else:
             print_xml_stream(raw_response_output)

mlcooper7
New Member

Thank you very much, perfect!

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...