Could someone aid with event field extractions and breaking?

[Splunk4]

Hi Everyone,

I am getting big single event through a python script from an API containing the performance data from an API but it is not autoextracting all the KV fields and i need to get those details to get the meaningful data.Also the timestamp is coming in epoch format.Below is the event format :

{'d': {'__count': '0', 'results': [{'ID': '6085', 'Name': 'device1', 'DisplayName': None, 'DisplayDescription': None, 'cpumfs': {'results': [{'ID': '6117', 'Timestamp': '1649157300', 'DeviceItemID': '6085', 'pct_im_Utilization': '4.0'}, {'ID': '6117', 'Timestamp': '1649157600', 'DeviceItemID': '6085', 'pct_im_Utilization': '1.0'}, {'ID': '6117', 'Timestamp': '1649157900', 'DeviceItemID': '6085', 'pct_im_Utilization': '4.0'}, {'ID': '6117', 'Timestamp': '1649158200', 'DeviceItemID': '6085', 'pct_im_Utilization': '1.0'}, {'ID': '6117', 'Timestamp': '1649158500', 'DeviceItemID': '6085', 'pct_im_Utilization': '4.0'}, {'ID': '6117', 'Timestamp': '1649158800', 'DeviceItemID': '6085', 'pct_im_Utilization': '1.0'}, {'ID': '6117', 'Timestamp': '1649159100', 'DeviceItemID': '6085', 'pct_im_Utilization': '4.0'}, {'ID': '6117', 'Timestamp': '1649159400', 'DeviceItemID': '6085', 'pct_im_Utilization': '1.0'}, {'ID': '6117', 'Timestamp': '1649159700', 'DeviceItemID': '6085', 'pct_im_Utilization': '4.0'}, {'ID': '6117', 'Timestamp': '1649160000', 'DeviceItemID': '6085', 'pct_im_Utilization': '1.0'}, {'ID': '6117', 'Timestamp': '1649160300', 'DeviceItemID': '6085', 'pct_im_Utilization': '4.0'}, {'ID': '6117', 'Timestamp': '1649160600', 'DeviceItemID': '6085', 'pct_im_Utilization': '1.0'}]}, 'memorymfs': {'results': [{'ID': '6118', 'Timestamp': '1649157300', 'DeviceItemID': '6085', 'im_Free': '2.809298944E9', 'pct_im_Utilization': '83.0702196963489'}, {'ID': '6118', 'Timestamp': '1649157600', 'DeviceItemID': '6085', 'im_Free': '2.741796864E9', 'pct_im_Utilization': '83.4770099337781'}, {'ID': '6118', 'Timestamp': '1649157900', 'DeviceItemID': '6085', 'im_Free': '2.784014336E9', 'pct_im_Utilization': '83.2225932482694'}, {'ID': '6118', 'Timestamp': '1649158200', 'DeviceItemID': '6085', 'im_Free': '2.739892224E9', 'pct_im_Utilization': '83.4884879350163'}, {'ID': '6118', 'Timestamp': '1649158500', 'DeviceItemID': '6085', 'im_Free': '2.812264448E9', 'pct_im_Utilization': '83.0523485718404'}, {'ID': '6118', 'Timestamp': '1649158800', 'DeviceItemID': '6085', 'im_Free': '2.747793408E9', 'pct_im_Utilization': '83.4408727427832'}, {'ID': '6118', 'Timestamp': '1649159100', 'DeviceItemID': '6085', 'im_Free': '2.808725504E9', 'pct_im_Utilization': '83.0736754386571'}, {'ID': '6118', 'Timestamp': '1649159400', 'DeviceItemID': '6085', 'im_Free': '2.744528896E9', 'pct_im_Utilization': '83.4605457900666'}, {'ID': '6118', 'Timestamp': '1649159700', 'DeviceItemID': '6085', 'im_Free': '2.804084736E9', 'pct_im_Utilization': '83.1016422674804'}, {'ID': '6118', 'Timestamp': '1649160000', 'DeviceItemID': '6085', 'im_Free': '2.740002816E9', 'pct_im_Utilization': '83.4878214704282'}, {'ID': '6118', 'Timestamp': '1649160300', 'DeviceItemID': '6085', 'im_Free': '2.7926528E9', 'pct_im_Utilization': '83.1705349587829'}, {'ID': '6118', 'Timestamp': '1649160600', 'DeviceItemID': '6085', 'im_Free': '2.736328704E9', 'pct_im_Utilization': '83.5099629050747'}]}}

In the above event , it is displaying CPU , memory utilization multiple times at different epoch times for each device.I have removed the trailing event containing data for other devices as it was exceeding the forum limit to post.I need to get the utilization data device wise.Please help on this.

Thanks

[richgalloway]

Where should the event(s) break?

What are the props.conf settings for this sourcetype?

[ITWhisperer]

What is the ask here? Do you want some SPL to extract some fields from the event (which has already been ingested) or so python script to break up the string into separate or smaller events or something else?

[Splunk4]

In the above event, there is device 1 mentioned and i want to get the CPU and memory utilization at the specified timestamps.For CPU utilization is starts with 'cpumfs' and under it contains multiple values at different timestamps like below :

cpumfs': {'results': [{'ID': '6117', 'Timestamp': '1649157300', 'DeviceItemID': '6085', 'pct_im_Utilization': '4.0'}, {'ID': '6117', 'Timestamp': '1649157600', 'DeviceItemID': '6085', 'pct_im_Utilization': '1.0'}, {'ID': '6117', 'Timestamp': '1649157900', 'DeviceItemID': '6085', 'pct_im_Utilization': '4.0'}

Similarly is the case for memory as below :

 'memorymfs': {'results': [{'ID': '6118', 'Timestamp': '1649157300', 'DeviceItemID': '6085', 'im_Free': '2.809298944E9', 'pct_im_Utilization': '83.0702196963489'}, {'ID': '6118', 'Timestamp': '1649157600', 'DeviceItemID': '6085', 'im_Free': '2.741796864E9', 'pct_im_Utilization': '83.4770099337781'}, {'ID': '6118', 'Timestamp': '1649157900', 'DeviceItemID': '6085', 'im_Free': '2.784014336E9', 'pct_im_Utilization': '83.2225932482694'}

But the device name is mentioned only once in the first line of event and not coming in the cpu and memory as mentioned above.It is coming like this in the first line of event :

{'d': {'__count': '0', 'results': [{'ID': '6085', 'Name': 'device1', 'DisplayName': None, 'DisplayDescription

I am not sure how to do the processing,whether through props or through python script

[richgalloway]

Given the desire to map the device name in the beginning to later entries, I think I'd write a Python script to process the data.

[Splunk4]

Can someone help me with the props configuration for this.I am trying to use below but its not working.

LINE_BREAKER = (,){"id":
INDEXED_EXTRACTIONS=json
KV_MODE=none
SHOULD_LINEMERGE=true
disabled=false
pulldown_type=true

I am trying to break the events starting with id