For some reason, I needed to share some data from an index with a different set of permissions.
After a bit of research, I found that the CLONE_SOURCETYPE option could help me with this stuff.
I created the required settings in props.conf and transforms.conf, and then pushed them to the IDXC layer.
At first glance, everything seemed fine, but then I discovered that CLONE_SOURCETYPE clones all events from the original sourcetype and redirects only a few to the new one.
Is that the intended behavior, or did I make serious mistakes in the configuration?
I expected to see only the events matching the REGEX in the original index.
props.conf
[vsi_file_esxi-syslog]
LINE_BREAKER = (\n)
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = \d{1,3}
TIME_PREFIX = ^<\d{1,3}>
TRANSFORMS-remove_trash = vsi_file_esxi-syslog_rt0, vsi_file_esxi-syslog_ke0
TRANSFORMS-route_events = general_file_esxi-syslog_re0
transforms.conf
[general_file_esxi-syslog_re0]
CLONE_SOURCETYPE = general_re_esxi-syslog
REGEX = FIREWALL-PKTLOG:
DEST_KEY = _MetaData:Index
FORMAT = general
WRITE_META = true
Yes, the behavior you're observing with CLONE_SOURCETYPE is expected. When you use CLONE_SOURCETYPE in Splunk, it creates a duplicate of every event that matches the props.conf stanza, regardless of the REGEX specified in the corresponding transforms.conf stanza. The REGEX is applied to the cloned event, not to determine whether an event should be cloned in the first place. This means that all events are cloned, and then the REGEX is used to modify or route the cloned events as specified.
https://community.splunk.com/t5/Getting-Data-In/Priority-precedence-in-props-conf/m-p/669047
CLONE_SOURCETYPE works on all events on which it is fired regardless of the REGEX value. In other words - you cannot limit its scope. If you assign a transform with CLONE_SOURCETYPE to a sourcetype, source or host, it will clone your event without any filtering.
And yes, the docs on CLONE_SOURCETYPE are a bit misleading and confusing.
Yes, the behavior you're observing with CLONE_SOURCETYPE is expected. When you use CLONE_SOURCETYPE in Splunk, it creates a duplicate of every event that matches the props.conf stanza, regardless of the REGEX specified in the corresponding transforms.conf stanza. The REGEX is applied to the cloned event, not to determine whether an event should be cloned in the first place. This means that all events are cloned, and then the REGEX is used to modify or route the cloned events as specified.
https://community.splunk.com/t5/Getting-Data-In/Priority-precedence-in-props-conf/m-p/669047