hi, I am querying an REST API to ingest the large JSON output. But facing issues with parsing JSON output. I am not interested int the metadata of the response I am only looking to ingest the data
{
"key": "9988",
"doc_count": 14
},
as an event. If I use the native reponsehandler Splunk is ingesting the data as one large even and stops at max events.
I tried using custom handler, but Splunk does not index any data. I tried to handle the out with custom sourcetype with no luck
` class DataHandler:
def __init__(self,**args):
pass
def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
if response_type == "json":
output = json.loads(raw_response_output)
for server in output["distinctValues"]["buckets"]:
print_xml_stream(json.dumps(server))
else:
print_xml_stream(raw_response_output)
`
JSON Response sample
{
"total": 15,
"querytime": 6,
"aggregations": {
"distinctValues": {
"doc_count": 0,
"sum_count": 0,
"buckets": [{
"key": "3412",
"doc_count": 15
}, {
"key": "5423",
"doc_count": 81
}, {
"key": "6543",
"doc_count": 16
}, {
"key": "6655",
"doc_count": 18
}, {
"key": "2344",
"doc_count": 10
}, {
"key": "1234",
"doc_count": 16
}, {
"key": "9898",
"doc_count": 10
}, {
"key": "4321",
"doc_count": 14
}, {
"key": "9988",
"doc_count": 14
}, {
"key": "3454",
"doc_count": 11
}, {
"key": "3242",
"doc_count": 14
}, {
"key": "9283",
"doc_count": 6
}, {
"key": "8472",
"doc_count": 1
}, {
"key": "9922",
"doc_count": 6
}, {
"key": "8293",
"doc_count": 5
}]
}
},
"results": []
}
