Splunk Enterprise

Is CLONE_SOURCETYPE works properly?

NoSpaces
Contributor

For some reason, I needed to share some data from an index with a different set of permissions.
After a bit of research, I found that the CLONE_SOURCETYPE option could help me with this stuff.
I created the required settings in props.conf and transforms.conf, and then pushed them to the IDXC layer.
At first glance, everything seemed fine, but then I discovered that CLONE_SOURCETYPE clones all events from the original sourcetype and redirects only a few to the new one.
Is that the intended behavior, or did I make serious mistakes in the configuration?
I expected to see only the events matching the REGEX in the original index.

 

props.conf

[vsi_file_esxi-syslog]
LINE_BREAKER = (\n)
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = \d{1,3}
TIME_PREFIX = ^<\d{1,3}>
TRANSFORMS-remove_trash = vsi_file_esxi-syslog_rt0, vsi_file_esxi-syslog_ke0
TRANSFORMS-route_events = general_file_esxi-syslog_re0
transforms.conf

[general_file_esxi-syslog_re0]
CLONE_SOURCETYPE = general_re_esxi-syslog
REGEX = FIREWALL-PKTLOG:
DEST_KEY = _MetaData:Index
FORMAT = general
WRITE_META = true

 

 

Labels (1)
0 Karma
1 Solution

kiran_panchavat
Champion

@NoSpaces 

Yes, the behavior you're observing with CLONE_SOURCETYPE is expected. When you use CLONE_SOURCETYPE in Splunk, it creates a duplicate of every event that matches the props.conf stanza, regardless of the REGEX specified in the corresponding transforms.conf stanza. The REGEX is applied to the cloned event, not to determine whether an event should be cloned in the first place. This means that all events are cloned, and then the REGEX is used to modify or route the cloned events as specified.

https://community.splunk.com/t5/Getting-Data-In/Priority-precedence-in-props-conf/m-p/669047 

https://community.splunk.com/t5/Getting-Data-In/Can-I-use-CLONE-SOURCETYPE-to-send-events-to-multipl... 

To clone only the events matching the REGEX to the new sourcetype and redirect them to the general index, while keeping all original events in the original index under the original sourcetype, you need to filter events before cloning. Unfortunately, Splunk’s CLONE_SOURCETYPE doesn’t natively support filtering during cloning.
 
You can use two transforms: one to filter out events that don’t match the REGEX and send them to nullQueue (discarding them from cloning), and another to clone and redirect the matching events. 
 
  • Events matching FIREWALL-PKTLOG: are cloned and routed to the general index.
  • The same matching events are dropped from the original index using nullQueue.
Did this help? If yes, please consider giving kudos, marking it as the solution, or commenting for clarification — your feedback keeps the community going!

View solution in original post

PickleRick
SplunkTrust
SplunkTrust

CLONE_SOURCETYPE works on all events on which it is fired regardless of the REGEX value. In other words - you cannot limit its scope. If you assign a transform with CLONE_SOURCETYPE to a sourcetype, source or host, it will clone your event without any filtering.

And yes, the docs on CLONE_SOURCETYPE are a bit misleading and confusing.

0 Karma

kiran_panchavat
Champion

@NoSpaces 

Yes, the behavior you're observing with CLONE_SOURCETYPE is expected. When you use CLONE_SOURCETYPE in Splunk, it creates a duplicate of every event that matches the props.conf stanza, regardless of the REGEX specified in the corresponding transforms.conf stanza. The REGEX is applied to the cloned event, not to determine whether an event should be cloned in the first place. This means that all events are cloned, and then the REGEX is used to modify or route the cloned events as specified.

https://community.splunk.com/t5/Getting-Data-In/Priority-precedence-in-props-conf/m-p/669047 

https://community.splunk.com/t5/Getting-Data-In/Can-I-use-CLONE-SOURCETYPE-to-send-events-to-multipl... 

To clone only the events matching the REGEX to the new sourcetype and redirect them to the general index, while keeping all original events in the original index under the original sourcetype, you need to filter events before cloning. Unfortunately, Splunk’s CLONE_SOURCETYPE doesn’t natively support filtering during cloning.
 
You can use two transforms: one to filter out events that don’t match the REGEX and send them to nullQueue (discarding them from cloning), and another to clone and redirect the matching events. 
 
  • Events matching FIREWALL-PKTLOG: are cloned and routed to the general index.
  • The same matching events are dropped from the original index using nullQueue.
Did this help? If yes, please consider giving kudos, marking it as the solution, or commenting for clarification — your feedback keeps the community going!
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

.conf25 Global Broadcast: Don’t Miss a Moment

Hello Splunkers, .conf25 is only a click away.  Not able to make it to .conf25 in person? No worries, you can ...

Observe and Secure All Apps with Splunk

 Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What's New in Splunk Observability - August 2025

What's New We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what is ...