topic Re: how can I keep the original meta when cloning the event. in Getting Data In

how can I keep the original meta when cloning the event.

sudha_krish — Tue, 15 Jul 2025 18:37:21 GMT

I'm cloning the event and before cloning extracting sourcetype to use later.

transforms.conf [copy_original_sourcetype] SOURCE_KEY = MetaData:Sourcetype REGEX = (.+) FORMAT = orig_sourcetype::$1 WRITE_META = true [clone_for_thirdparty] SOURCE_KEY = _MetaData:Index REGEX = ^test_np$ DEST_KEY = MetaData:Sourcetype CLONE_SOURCETYPE = data_to_thirdparty WRITE_META = true [sourcetype_raw_updated] SOURCE_KEY=MetaData:orig_sourcetype REGEX=^orig_sourcetype::(.*)$ FORMAT = $1##$0 DEST_KEY=_raw

But when I try to retrieve extracted original value I'm getting nothing. Is there any way to persist original sourcetype ?

@PickleRick @isoutamo @gcusello

Re: how can I keep the original meta when cloning the event.

sudha_krish — Tue, 15 Jul 2025 18:39:27 GMT

I just want to add the orig_sourcetype to the cloned event with original sourcetype value.

Re: how can I keep the original meta when cloning the event.

livehybrid — Tue, 15 Jul 2025 19:20:54 GMT

Hey @sudha_krish

Please avoid calling out specific users on here - it wont help get your question answered.

Please could you also share your props.conf config?

I dont think this is correct:

SOURCE_KEY=MetaData:orig_sourcetype

Can you try:

[copy_original_sourcetype] SOURCE_KEY = MetaData:Sourcetype REGEX = (.+) FORMAT = orig_sourcetype::$1 WRITE_META = true [clone_for_thirdparty] SOURCE_KEY = MetaData:Index REGEX = ^test_np$ DEST_KEY = MetaData:Sourcetype CLONE_SOURCETYPE = data_to_thirdparty WRITE_META = true [sourcetype_raw_updated] INGEST_EVAL = _raw=_raw." orig_sourcetype=".orig_sourcetype

Do you want orig_sourcetype adding to the raw text value?

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: how can I keep the original meta when cloning the event.

isoutamo — Tue, 15 Jul 2025 20:55:27 GMT

What is in your props.conf?

Re: how can I keep the original meta when cloning the event.

sudha_krish — Wed, 16 Jul 2025 05:05:04 GMT

Here is my props and transforms

props.conf [default] TRANSFORMS-save_original_sourcetype = save_original_sourcetype TRANSFORMS-clone_for_thirdparty = clone_for_thirdparty [data_to_thirdparty] SHOULD_LINEMERGE = true TRANSFORMS-updateFields = sourcetype_raw_updated TRANSFORMS-route_thirdparty = route_thirdparty transforms.conf # 1. Save original sourcetype as a field (for use in the clone) [save_original_sourcetype] SOURCE_KEY = MetaData:Sourcetype REGEX = (.+) FORMAT = orig_sourcetype::$1 WRITE_META = true # 2. Clone only events from vme_ops_prod to sourcetype=data_to_thirdparty [clone_for_thirdparty] SOURCE_KEY = _MetaData:Index REGEX = ^test_np$ DEST_KEY = MetaData:Sourcetype CLONE_SOURCETYPE = data_to_thirdparty WRITE_META = true # 3. Meta-data transforms for the clone [sourcetype_raw_updated] SOURCE_KEY=MetaData:orig_sourcetype REGEX=^orig_sourcetype::(.*)$ FORMAT = $1##$0 DEST_KEY=_raw # 4. Route the cloned event ONLY to thirdparty [route_thirdparty] SOURCE_KEY = _MetaData:Index REGEX = (^test_np.*) DEST_KEY = _TCP_ROUTING FORMAT = dev_thirdparty

I'm sending logs (no cooked data) to thirdpartyserver over TCP without disturbing existing flow. so I just cloned the event and adding original sourcetype to the cloned event and sending to thirdparty output group.

now the issue here is I can't find the original source type.

Re: how can I keep the original meta when cloning the event.

PickleRick — Wed, 16 Jul 2025 07:26:11 GMT

Transform classes are called in alphabetical order. And please don't call out specific people for help. That's rude.