I have the need to change the sourcetype of certain logs on a per-event basis, then apply further changes on the new sourcetype in props.conf after the sourcetype change. For example:
[set_new_sourcetype] DEST_KEY = MetaData:Sourcetype REGEX = some_regex FORMAT = sourcetype::my:new:sourcetype
[old:sourcetype] TRANSFORMS-sourcetype_change = set_new_sourcetype [my:new:sourcetype] SEDCMD-whatever = s/change/that/g
However, the SEDCMD does not work on the new sourcetype. If I move the SEDCMD to the original sourcetype it works. Is there a workaround for this? If not, what's the point of being able to change the sourcetype on a per-event basis if you can't do anything with it afterwards?
Please have a try like
[old:sourcetype] TRANSFORMS-sourcetype_change = set_new_sourcetype,substitute_whatever
[set_new_sourcetype] DEST_KEY = MetaData:Sourcetype REGEX = some_regex FORMAT = sourcetype::my:new:sourcetype [substitute_whatever] REGEX = (findSomeKey) FORMAT = substitutedValue DEST_KEY = _raw
Hi @koshyk, thanks for the reply. I'm not sure this will work in my case. I'm setting the sourcetype based on one string that is only common to those logs, however the string I need removed from the new sourcetype is common to the logs in both the new and old sourcetypes. The way this is written, I believe it will affect both of my sourcetypes, correct?
That is correct. So you essentially need "almost duplicate" data in two sourcetypes.
If that's the case, in the above logic, do as two lines. Ensure that the TRANSFORM names are increasing order of ASCII . Something like below
[old:sourcetype] TRANSFORMS-1sourcetype_change = set_new_sourcetype TRANSFORMS-2substitute_whatever = substitute_whatever