topic Re: Separate the array value in the different event. in Splunk Search

Separate the array value in the different event.

parthiban — Mon, 11 Mar 2024 06:15:24 GMT

Hi team,

I mentioned that the payload field contains the entity-internal-id and lead-id in an array format. I want to print a separate event with one lead and one entity internal id present, and the rest of the values will be printed in the next event, respectively. Kindly suggest here.

correlation_id: ********
custom_attributes: { [-]
     campaign-id: ****
     campaign-name: ******
     country:
     entity-internal-id: [ [-]
       12345678
       87654321
     ]
     lead-id: [ [-]
       11112222
       33334444
     ]
     marketing-area: *****
     record_count:
     root-entity-id: 2
}

Re: Separate the array value in the different event.

ITWhisperer — Mon, 11 Mar 2024 10:34:37 GMT

This looks like it might be JSON - if so, please provide your example (anonymised) event(s) in raw/unformatted form, i.e. valid JSON syntax, preferably in a code block </>

Re: Separate the array value in the different event.

parthiban — Mon, 11 Mar 2024 11:44:37 GMT

Hi @ITWhisperer
Here the raw format

{"message_type": "INFO", "processing_stage": "XXXXX", "message": "XXXXXX", "correlation_id": "XXXXXX", "error": "", "invoker_agent": "XXXXXX", "invoked_component": "XXXXXX, "request_payload": "", "response_details": "", "invocation_timestamp": "XXXXX", "response_timestamp": "XXXXX", "original_source_app": "XXXX", "AAAA": "", "retry_attempt": "1", "custom_attributes": {"entity-internal-id": ["12345678", "9876543", "2341234"], "root-entity-id": "3", "campaign-id": "XXXX", "campaign-name": "XXXXX", "marketing-area": "CCCC", "lead-id": ["000000", "1111111", "3333333"], "record_count": "", "country": ""}}

Re: Separate the array value in the different event.

ITWhisperer — Mon, 11 Mar 2024 15:15:22 GMT

Try something like this

| spath custom_attributes output=custom_attributes | spath input=custom_attributes | eval combined=mvzip('entity-internal-id{}','lead-id{}') | mvexpand combined | eval entity_internal_id = mvindex(split(combined,","),0) | eval lead_id = mvindex(split(combined,","),1)

Re: Separate the array value in the different event.

yuanliu — Wed, 13 Mar 2024 01:03:21 GMT

If your JSON-compliant data contains two arrays that has to be mapped externally, your developers have committed the highest design crime. If you have any influence over development team, beg them, implore them, curse them to change custom_attributes to something like

{"root-entity-id":"3","campaign-id":"XXXX","campaign-name":"XXXXX","marketing-area":"CCCC","record_count":"","country":"","id_array":[{"internal":"12345678","lead":"000000"},{"internal":"9876543","lead":"1111111"},{"internal":"2341234","lead":"3333333"}]}

This way, data processing (in any language, not just Splunk) will be much cleaner. More importantly, downstream programmers such as yourself will not need to have this vertical knowledge about implied semantics.

No implied semantics is one of the most important advantages for people to adopt structured data formats such as JSON. This means lower maintenance cost in the future.

Re: Separate the array value in the different event.

parthiban — Wed, 13 Mar 2024 08:22:48 GMT

Thank you for your support @ITWhisperer , the given code is working as expected.