Separate the array value in the different event.

[parthiban]

Hi team,

I mentioned that the payload field contains the entity-internal-id and lead-id in an array format. I want to print a separate event with one lead and one entity internal id present, and the rest of the values will be printed in the next event, respectively. Kindly suggest here.

correlation_id: ********
custom_attributes: { [-]
     campaign-id: ****
     campaign-name: ******
     country:
     entity-internal-id: [ [-]
       12345678
       87654321
     ]
     lead-id: [ [-]
       11112222
       33334444
     ]
     marketing-area: *****
     record_count:
     root-entity-id: 2
}

[ITWhisperer]

This looks like it might be JSON - if so, please provide your example (anonymised) event(s) in raw/unformatted form, i.e. valid JSON syntax, preferably in a code block </>

[parthiban]

Hi @ITWhisperer 
Here the raw format

{"message_type": "INFO", "processing_stage": "XXXXX", "message": "XXXXXX", "correlation_id": "XXXXXX", "error": "", "invoker_agent": "XXXXXX", "invoked_component": "XXXXXX, "request_payload": "", "response_details": "", "invocation_timestamp": "XXXXX", "response_timestamp": "XXXXX", "original_source_app": "XXXX", "AAAA": "", "retry_attempt": "1", "custom_attributes": {"entity-internal-id": ["12345678", "9876543", "2341234"], "root-entity-id": "3", "campaign-id": "XXXX", "campaign-name": "XXXXX", "marketing-area": "CCCC", "lead-id": ["000000", "1111111", "3333333"], "record_count": "", "country": ""}}

[yuanliu]

If your JSON-compliant data contains two arrays that has to be mapped externally, your developers have committed the highest design crime.  If you have any influence over development team, beg them, implore them, curse them to change custom_attributes to something like

{"root-entity-id":"3","campaign-id":"XXXX","campaign-name":"XXXXX","marketing-area":"CCCC","record_count":"","country":"","id_array":[{"internal":"12345678","lead":"000000"},{"internal":"9876543","lead":"1111111"},{"internal":"2341234","lead":"3333333"}]}

This way, data processing (in any language, not just Splunk) will be much cleaner.  More importantly, downstream programmers such as yourself will not need to have this vertical knowledge about implied semantics.

No implied semantics is one of the most important advantages for people to adopt structured data formats such as JSON.  This means lower maintenance cost in the future.

[ITWhisperer · Accepted solution]

Try something like this

| spath custom_attributes output=custom_attributes
| spath input=custom_attributes
| eval combined=mvzip('entity-internal-id{}','lead-id{}')
| mvexpand combined
| eval entity_internal_id = mvindex(split(combined,","),0)
| eval lead_id = mvindex(split(combined,","),1)

[parthiban]

Thank you for your support @ITWhisperer , the given code is working as expected.