Getting Data In

How to break one event to multiple events using my univarsal forwarder props.conf?

bapun18
Communicator

Hi

I wanted to break the line from {"id" so that splunk will treat it as a new event from {"id from below event, I have mentioned the props.conf and the event, please find the same and let me know in case of any concerns.

 

INDEXED_EXTRACTIONS = JSON
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
SEGMENTATION = iso8601
#TIME_FORMAT=%YYYY-%MM-%DDT%H:%M:%SZ
TIMESTAMP_FIELDS = started_on
TRUNCATE = 0
category = Ver. 1

 

 

Labels (2)
0 Karma
1 Solution

bapun18
Communicator

Issue resolved I have to change the JSON output data with removing of {

View solution in original post

0 Karma

bapun18
Communicator

Issue resolved I have to change the JSON output data with removing of {

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Short answer is - you can't.

Long answer is - as soon as the input data stream is split into single events by means of line breakers, event breakers and line merging, you have a single event and that's it - that event is getting processed as a whole. It's getting passed through all ingest-time transforms as a single event. You can modify it, you can parse indexed fields, you can add metadata but it's still that single event.

All solutions - provided by @somesoni2 as well as other ones fall into one of two categories

1) modifying the event splitting rules so the forwarder does not pick the whole json at once but picks up parts of it individually - involves tweaking line/event breakers so they match your "subevent" boundaries

2) splitting your events in search time - you can parse json using spath, then mvexpand, overwrite _time but you can't use this info for initial timerange selection.

The only other possibility I see is modifying your event before even ingesting it to splunk - possibly by means of external script or modular input.

bapun18
Communicator

Yeah I have tried but seems not working, still need more help

0 Karma

bapun18
Communicator
{"_links":{"prev":"","self":"/api/v1.1/instances?start=0\u0026limit=100\u0026date_from=2022-03-10T03:57:46.147806Z","next":""},"results":[{"id":"abcd","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJBI2DTI9VO5k9CTCyLMa8C6wmwMBAMik","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T15:05:02.8Z","ended_on":"2022-03-10T15:05:42.517Z","created_on":"2022-03-10T15:05:02.614Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T15:05:42.535Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcde","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ9L66VYZNH6wyPXPJRD3bU1nGTLbKM4l","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T13:35:03.915Z","ended_on":"2022-03-10T13:35:35.962Z","created_on":"2022-03-10T13:35:03.774Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T13:35:35.98Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcdef","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ8Y70GXAZS30443lVF9SQKm7fQZSxVDB","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T13:05:03.501Z","ended_on":"2022-03-10T13:05:50.201Z","created_on":"2022-03-10T13:05:03.187Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T13:05:50.221Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcdr","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ8MPQ3G8DQ1UX2NAlN3vp3YWagVgw8Xt","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T12:50:03.703Z","ended_on":"2022-03-10T12:50:38.812Z","created_on":"2022-03-10T12:50:03.549Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T12:50:38.832Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abcs","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ7W2P3ZGNN4qGTX33OxT8ZcaoszxLUIR","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"success","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T12:15:16.115Z","ended_on":"2022-03-10T12:15:31.396Z","created_on":"2022-03-10T12:15:15.955Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T12:15:31.416Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abvf","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ6PSOKL0LZ4SuwRI0eCTYovpUSqC1SMC","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"failed","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T11:20:03.037Z","ended_on":"2022-03-10T11:20:04.44Z","created_on":"2022-03-10T11:20:02.86Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T11:20:04.443Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"abvk","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"01VJ62UA4LYQU0od77lbUDPiDWbXIu0DdcI","schema_id":"khkhllijj","version":"1.0.0","name":"Initialize Case","type":"generic.workflow","base_type":"workflow","properties":{"atomic":{"is_atomic":false},"delete_workflow_instance":false,"description":"Run XDR new alert event workflow.\nValidate, Enrich, Template, Reputation, Past Tickets, Aggregate","display_name":"Initialize Case","runtime_user":{"target_default":true},"target":{"execute_on_target_group":true,"target_group":{"run_on_all_targets":false,"selected_target_types":["01JYJ0OOD9O7P2lkhNF1LsJrTonSOcI3AXu"],"target_group_id":"01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk","use_criteria":{"choose_target_using_algorithm":"choose_first_with_matching_criteria","conditions":[{"left_operand":"$targetgroup.01UTUCNGTOL5553AD59jsiGR7eeOJYLuiPk.common.display_name$","operator":"eqi","right_operand":"Splunk_Target"}]}}}},"status":{"state":"failed","prev_state":"running"},"started_by":"akm+klooperate@abc.com","started_on":"2022-03-10T10:50:04.264Z","ended_on":"2022-03-10T10:50:05.036Z","created_on":"2022-03-10T10:50:03.964Z","created_by":"akm+klooperate@abc.com","updated_on":"2022-03-10T10:50:05.052Z","updated_by":"akm+klooperate@abc.com","owner":"kklerkin+klooperate@abc.com"},{"id":"aljd","definition_id":"ajgjgjkgkukugkugu","root_workflow_id":"
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...