Hello,
I am experiencing unexpected behavior with the CLONE_SOURCETYPE attribute in transforms. When I use CLONE_SOURCETYPE, Splunk ends up indexing both copies of the cloned event. I am using "DEST_KEY = _SYSLOG_ROUTING" within the same stanza as CLONE_SOURCETYPE and it will route a copy of the cloned sourcetype to the syslog output processor but also indexes the cloned events.
Per the documentation for transforms detailing the values for "keys": "NOTE: Any KEY (field name) prefixed by '_' is not indexed by Splunk, in general." I am interpreting that as meaning that when using "DEST_KEY = _SYSLOG_ROUTING" the sourcetype should not be indexed.
Is this a bug in CLONE_SOURCETYPE?
Here is the config, I changed naming to protect privacy.
1. First I am applying the transforms named "clone_sourcetype" which makes a clone of all Windows Event Logs with a new sourcetype named "SIEM_FORMAT"
##PROPS##
[source::WinEventLog:*]
TRANSFORMS-WinEventLog = clone_sourcetype
##TRANSFORMS-1##
[clone_sourcetype]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
CLONE_SOURCETYPE = SIEM_FORMAT
2. Second I am taking the new sourcetype "SIEM_FORMAT" and am applying SEDCMD and LINEMERGE to merge the multiline Windows events into a single line event. I also apply another transform called "SIEM_syslog" which applies the output stanza "send_syslog_to_SIEM".
##PROPS##
[SIEM_FORMAT]
SEDCMD-rmlines=s/[\n\r\t]/ /g
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = ((.+)\d+\/\d+\/\d+\s+\d+:\d+:\d+\s+([aApPmM]{2}))
TRANSFORMS-output = SIEM_syslog
##TRANSFORMS##
[SIEM_syslog]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = send_syslog_to_SIEM
##OUTPUTS##
[syslog:send_syslog_to_SIEM]
server = x.x.x.x:514
type = tcp
priority = NO_PRI
Everything works perfectly, I am seeing the events on the SIEM side formatted the way I need them except Splunk is indexing both the original and cloned sourcetype.
Any ideas as to why this is occuring?
... View more
