To recap, the problem is that we have a source whose events need to be split and end up in a certain target format. In this particular case, this is done on a universal forwarder, but the solution applies to a source local to a Splunk indexer too.
1) In inputs.conf , identify the sourcetype as intermediate_sourcetype_1 .
2) In props.conf (with force_local_processing = true ), assign to intermediate_sourcetype_1 any common _raw transformation to a SEDCMD , then specify three TRANSFORMS which we'll call transform-clone-1, transform-clone-2, transform-drop .
3) In transforms.conf , define transform-drop as REGEX = .* , DEST_KEY = queue , FORMAT = nullQueue . This transform simply drops the event.
4) Still in transforms.conf , define transform-clone-1 as REGEX = .* , DEST_KEY = _raw , FORMAT = $0 , CLONE_SOURCETYPE = intermediate_sourcetype_2A . transform-clone-2 is the same except that CLONE_SOURCETYPE = intermediate_sourcetype_2B .
5) Back in props.conf , assign appropriate SEDCMD and TRANSFORMS to intermediate_sourcetype_2A and intermediate_sourcetype_2B . Make sure to conclude the TRANSFORMS set with transform-switch-sourcetype .
6) Finally, back in transforms.conf , define transform-switch-sourcetype as SOURCE_KEY = MetaData:Sourcetype , REGEX = .* , DEST_KEY = MetaData:Sourcetype , FORMAT = sourcetype::target_sourcetype . This transform simply switches the event's sourcetype.
It's important that props.conf not rely on source:: stanzas to process the events, because that stanza would apply to the cloned events as well as to the original event, resulting in multiple applications of SEDCMD and TRANSFORMS . That is unlikely to yield the desired results except in really odd circumstances.
A note of caution: keep backups of inputs.conf , props.conf and transforms.conf appearing in the universal forwarder's /opt/splunkforwarder/etc/apps/_server_app_<server_class>/local because Splunk Web will wipe them when you change the input configuration—if you edit directly in that folder. The workaround is to build your inputs.conf , props.conf , and transforms.conf in /opt/splunk/etc/deployment-apps/_server_app_<server_class>/local on the main Splunk instance. The only caveat I've had with this is that there is seemingly no way to "refresh" the forwarder from Splunk Web (you'd expect that in Settings: (Distributed environment) Forwarder management ); you must use the command line and issue splunk reload deploy-server .
... View more