Getting Data In

How to break one event to multiple events using my univarsal forwarder props.conf?

bapun18
Communicator

Hi

I wanted to break the line from {"id" so that splunk will treat it as a new event from {"id from below event, I have mentioned the props.conf and the event, please find the same and let me know in case of any concerns.

 

INDEXED_EXTRACTIONS = JSON
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
SEGMENTATION = iso8601
#TIME_FORMAT=%YYYY-%MM-%DDT%H:%M:%SZ
TIMESTAMP_FIELDS = started_on
TRUNCATE = 0
category = Ver. 1

 

 

Labels (2)
0 Karma
1 Solution

bapun18
Communicator

Issue resolved I have to change the JSON output data with removing of {

View solution in original post

0 Karma

bapun18
Communicator

Issue resolved I have to change the JSON output data with removing of {

0 Karma

PickleRick
Ultra Champion

Short answer is - you can't.

Long answer is - as soon as the input data stream is split into single events by means of line breakers, event breakers and line merging, you have a single event and that's it - that event is getting processed as a whole. It's getting passed through all ingest-time transforms as a single event. You can modify it, you can parse indexed fields, you can add metadata but it's still that single event.

All solutions - provided by @somesoni2 as well as other ones fall into one of two categories

1) modifying the event splitting rules so the forwarder does not pick the whole json at once but picks up parts of it individually - involves tweaking line/event breakers so they match your "subevent" boundaries

2) splitting your events in search time - you can parse json using spath, then mvexpand, overwrite _time but you can't use this info for initial timerange selection.

The only other possibility I see is modifying your event before even ingesting it to splunk - possibly by means of external script or modular input.

somesoni2
Revered Legend

bapun18
Communicator

Yeah I have tried but seems not working, still need more help

0 Karma

bapun18
Communicator
{"_links":{"prev":"","self":"/api/v1.1/instances?start=0\u0026limit=100\u0026date_from=2022-03-10T03:57:46.147806Z","next":""},"results":[{"id":"abcd","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJBI2DTI9VO5k9CTCyLMa8C6wmwMBAMik","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T15:05:02.8Z","ended_on":"2022-03-10T15:05:42.517Z","created_on":"2022-03-10T15:05:02.614Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T15:05:42.535Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcde","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ9L66VYZNH6wyPXPJRD3bU1nGTLbKM4l","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T13:35:03.915Z","ended_on":"2022-03-10T13:35:35.962Z","created_on":"2022-03-10T13:35:03.774Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T13:35:35.98Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcdef","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ8Y70GXAZS30443lVF9SQKm7fQZSxVDB","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T13:05:03.501Z","ended_on":"2022-03-10T13:05:50.201Z","created_on":"2022-03-10T13:05:03.187Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T13:05:50.221Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcdr","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ8MPQ3G8DQ1UX2NAlN3vp3YWagVgw8Xt","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T12:50:03.703Z","ended_on":"2022-03-10T12:50:38.812Z","created_on":"2022-03-10T12:50:03.549Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T12:50:38.832Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcs","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ7W2P3ZGNN4qGTX33OxT8ZcaoszxLUIR","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T12:15:16.115Z","ended_on":"2022-03-10T12:15:31.396Z","created_on":"2022-03-10T12:15:15.955Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T12:15:31.416Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abvf","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ6PSOKL0LZ4SuwRI0eCTYovpUSqC1SMC","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"failed","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T11:20:03.037Z","ended_on":"2022-03-10T11:20:04.44Z","created_on":"2022-03-10T11:20:02.86Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T11:20:04.443Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abvk","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ62UA4LYQU0od77lbUDPiDWbXIu0DdcI","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"failed","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T10:50:04.264Z","ended_on":"2022-03-10T10:50:05.036Z","created_on":"2022-03-10T10:50:03.964Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T10:50:05.052Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"aljd","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"
0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...