How to break one event to multiple events using my univarsal forwarder props.conf?

[bapun18]

Hi

I wanted to break the line from {"id" so that splunk will treat it as a new event from {"id from below event, I have mentioned the props.conf and the event, please find the same and let me know in case of any concerns.

INDEXED_EXTRACTIONS = JSON
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
SEGMENTATION = iso8601
#TIME_FORMAT=%YYYY-%MM-%DDT%H:%M:%SZ
TIMESTAMP_FIELDS = started_on
TRUNCATE = 0
category = Ver. 1

[bapun18 · Accepted solution]

Issue resolved I have to change the JSON output data with removing of {

[PickleRick]

Short answer is - you can't.

Long answer is - as soon as the input data stream is split into single events by means of line breakers, event breakers and line merging, you have a single event and that's it - that event is getting processed as a whole. It's getting passed through all ingest-time transforms as a single event. You can modify it, you can parse indexed fields, you can add metadata but it's still that single event.

All solutions - provided by @somesoni2 as well as other ones fall into one of two categories

1) modifying the event splitting rules so the forwarder does not pick the whole json at once but picks up parts of it individually - involves tweaking line/event breakers so they match your "subevent" boundaries

2) splitting your events in search time - you can parse json using spath, then mvexpand, overwrite _time but you can't use this info for initial timerange selection.

The only other possibility I see is modifying your event before even ingesting it to splunk - possibly by means of external script or modular input.

[somesoni2]

There are several previous posts which address this issue. Here are some of these posts:

https://community.splunk.com/t5/Getting-Data-In/How-to-split-a-json-array-into-multiple-events-with-...

https://community.splunk.com/t5/Getting-Data-In/how-to-split-the-json-array-into-multiple-new-events...

[bapun18]

Yeah I have tried but seems not working, still need more help

[bapun18]

{"_links":{"prev":"","self":"/api/v1.1/instances?start=0\u0026limit=100\u0026date_from=2022-03-10T03:57:46.147806Z","next":""},"results":[{"id":"abcd","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJBI2DTI9VO5k9CTCyLMa8C6wmwMBAMik","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T15:05:02.8Z","ended_on":"2022-03-10T15:05:42.517Z","created_on":"2022-03-10T15:05:02.614Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T15:05:42.535Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcde","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ9L66VYZNH6wyPXPJRD3bU1nGTLbKM4l","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T13:35:03.915Z","ended_on":"2022-03-10T13:35:35.962Z","created_on":"2022-03-10T13:35:03.774Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T13:35:35.98Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcdef","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ8Y70GXAZS30443lVF9SQKm7fQZSxVDB","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T13:05:03.501Z","ended_on":"2022-03-10T13:05:50.201Z","created_on":"2022-03-10T13:05:03.187Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T13:05:50.221Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcdr","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ8MPQ3G8DQ1UX2NAlN3vp3YWagVgw8Xt","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T12:50:03.703Z","ended_on":"2022-03-10T12:50:38.812Z","created_on":"2022-03-10T12:50:03.549Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T12:50:38.832Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcs","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ7W2P3ZGNN4qGTX33OxT8ZcaoszxLUIR","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T12:15:16.115Z","ended_on":"2022-03-10T12:15:31.396Z","created_on":"2022-03-10T12:15:15.955Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T12:15:31.416Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abvf","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ6PSOKL0LZ4SuwRI0eCTYovpUSqC1SMC","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"failed","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T11:20:03.037Z","ended_on":"2022-03-10T11:20:04.44Z","created_on":"2022-03-10T11:20:02.86Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T11:20:04.443Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abvk","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ62UA4LYQU0od77lbUDPiDWbXIu0DdcI","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"failed","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T10:50:04.264Z","ended_on":"2022-03-10T10:50:05.036Z","created_on":"2022-03-10T10:50:03.964Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T10:50:05.052Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"aljd","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"