How to modify sourcetype and host based off of the new sourcetype

[nateloepker]

Hello,

I'm attempting to change the sourcetype and host on a single event. The tricky part is I want the second transform based on the change from the first transform

For Example,

My data comes in as

index=main

host=heavy_forwarder

sourcetype=aws:logbucket

I want the data to change to

index=main

host=amazonfsx.host

sourcetype=XmlWinEventLog

The catch is that I have other sourcetypes coming in as aws:logbucket and getting transformed to various other sourcetypes (cloudtrail, config, etc). On these events I do not want to run the regex to change the host value

If I have a props.conf file that states

TRANSFORMS-modify_data = aws_fsx_sourcetype, aws_fsx_host

And a transforms.conf of

[aws_fsx_sourcetype]
SOURCE_KEY = MetaData:Source
REGEX = ^source::s3:\/\/fsxbucket\/.*
FORMAT = sourcetype::XmlWinEventLog
DEST_KEY = MetaData:Sourcetype

[aws_fsx_host]
REGEX = <Computer>([^.<]+).*?<\/Computer>
FORMAT = host::$1
DEST_KEY = MetaData:Host

I'm worried this will have unexpected results on the other sourcetypes that aws:logucket has, like cloudtrail and config.

If I break it out with two separate transforms, like this

TRANSFORMS-modify_data = aws_fsx_sourcetype

TRANSFORMS-modify_data2 = aws_fsx_host

I'm worried the typing pipeline won't see the second transform.

What is the most effective way to accomplish this?

Thanks,
Nate

[yuanliu]

Altering host field is one of least desirable alterations but it has to be done from time to time.  In your case, you probably have to use a calculated field.

[nateloepker]

I'm going to try that. I'm seeing now the Windows App has a default transformation called "WinEventXmlHostOverride" that will override the host with the "Computer" Xml value. Do you see any negative from doing this at search-time rather than index-time?

[yuanliu]

Calculated field and transforms are also search time.  The general advice is to be very careful in making index time calculations. (Except filtering down.)