Extract event fields in desired format

[rantravee]

hi,

I'd need some hints regarding the propertiesI should have in in props.conf and transforms.conf so that I have data in the needed format. So I within my scripted input I print to standard output a json object of the following format

{

"statistics" :[{stats_resource_json_obj_1},....,{stats_resource_json_obj_n}]

}

each stats_resource_json_obj_i has the folowing format
{
"id":value,
............
"stat_i":"val_i"
}
I would like to have an event for each stats-resource-json-obj and to have recognized/extracted as fields (at search time) every pair of key/value within the object .

Thanks a lot

[alacercogitatus]

In your props.conf, make sure that your KV_MODE is set to JSON. In order to split up the events, since you have control of the script, I'd rework the script to print a new json object on each line, Splunk should handle the rest.

[rantravee]

The json obj comes in the format described in the question from a network resource :

resp, content = http.request(statistics_url,
method='POST',
headers={'Content-Type': 'application/json', 'charset':'UTF-8','Connection':'keep-alive', 'Host':'theHost'}, body=json_body)

and then I call printResponseToSplunk(), passing the content object to it

[rantravee]

Thanks for the channel hint. I was unware of that.

[alacercogitatus]

Can you paste the entire script to pastebin please? I want to see how you are crafting the string that you then dump with the json object. Don't forget we can help you out in Real Time on the IRC #splunk channel on efnet.

[rantravee]

def printResponseToSplunk(self,s):
jdata = json.loads(s)

# Augment json object with additional information
stats = jdata[JSON_STATS_OBJ_NAME]

for innerOBj in stats
innerOBj[JSON_CHASSIS_KEY]=self.getHost()
print (json.dumps(innerOBj))
sys.stdout.flush()

[alacercogitatus]

I would need to see a pastebin of you code to see what you are doing to accurately diagnose it.

[rantravee]

How could I achieve that ? I realize that my question may sound ridiculous , but I just recently crushed Pyhton Programming.

[alacercogitatus]

remove the single quotes from the field names.

[rantravee]

It seems Splunk is incapable of automatically extracting the fields .Only the default fields are extracted . I see a long string ,containing all the key/pair values . Something like

{'stat_1': 0, 'stat_2': 0, 'stat_3': 0, 'stat_4': 0, 'stat_4': 0, ....., 'stat_n': 0 }