Applying changes to logs whose sourcetype has been changed on a per-event basis

[ehowardl3]

I have the need to change the sourcetype of certain logs on a per-event basis, then apply further changes on the new sourcetype in props.conf after the sourcetype change. For example:

transforms.conf:

 [set_new_sourcetype]
 DEST_KEY = MetaData:Sourcetype
 REGEX = some_regex
 FORMAT = sourcetype::my:new:sourcetype

props.conf:

[old:sourcetype]
TRANSFORMS-sourcetype_change = set_new_sourcetype

[my:new:sourcetype]
SEDCMD-whatever = s/change/that/g

However, the SEDCMD does not work on the new sourcetype. If I move the SEDCMD to the original sourcetype it works. Is there a workaround for this? If not, what's the point of being able to change the sourcetype on a per-event basis if you can't do anything with it afterwards?

Thanks

[koshyk]

Please have a try like

props.conf

[old:sourcetype]
TRANSFORMS-sourcetype_change = set_new_sourcetype,substitute_whatever

transforms.conf

[set_new_sourcetype]
 DEST_KEY = MetaData:Sourcetype
 REGEX = some_regex
 FORMAT = sourcetype::my:new:sourcetype

  [substitute_whatever]
  REGEX = (findSomeKey)
  FORMAT = substitutedValue
  DEST_KEY = _raw

[ehowardl3]

Hi @koshyk, thanks for the reply. I'm not sure this will work in my case. I'm setting the sourcetype based on one string that is only common to those logs, however the string I need removed from the new sourcetype is common to the logs in both the new and old sourcetypes. The way this is written, I believe it will affect both of my sourcetypes, correct?

[koshyk]

That is correct. So you essentially need "almost duplicate" data in two sourcetypes.
If that's the case, in the above logic, do as two lines. Ensure that the TRANSFORM names are increasing order of ASCII . Something like below

 [old:sourcetype]
 TRANSFORMS-1sourcetype_change = set_new_sourcetype
 TRANSFORMS-2substitute_whatever = substitute_whatever