I have a customer with a nightmare syslog server environment -- different sourcetypes in different log files on different syslog servers, shared unqualified hostnames used in different data centers, some logs have FQDNs, some don't, etc.
My understanding is that the order of precedence for TRANSFORMS is source:: overwrites both sourcetype and host:: stanzas. host:: overwrites sourcetype stanzas.
So... I have TRANSFORMS stanzas applied to each source:: stanza to put the appropriate data into the correct sourcetype. I then apply index and host metadata TRANSFORMS to each of the sourcetype stanzas.
But for some reason, the host and index TRANSFORMS don't seem to get applied once an event has had a TRANSFORM applied in a source:: stanza. Is that expected behavior or are there limitations to metadata rewrites that they must occur only on the stanza with the highest precedence for a particular event?
The way you are trying to achieve I tried in my lab environment and it is not working which means props and transforms once applied then data will not route again through pipeline to modify host and index. However you can achieve sourcetype override, hostname override and index override with below configuration.
I have applied below configuration on
WinEventLog:Security source and playing with EventCode 4688 to filter data using REGEX
[source::WinEventLog:Security] TRANSFORMS-sthostidx = test_st, test_host, test_idx
[test_st] DEST_KEY = MetaData:Sourcetype REGEX = EventCode\=4688 FORMAT = sourcetype::test_sourcetype [test_host] SOURCE_KEY = MetaData:Sourcetype DEST_KEY = MetaData:Host REGEX = test_sourcetype FORMAT = host::testhost [test_idx] SOURCE_KEY = MetaData:Host DEST_KEY = _MetaData:Index REGEX = testhost FORMAT = windows
I hope this will help you.
@harsmarvania57 - What is the need here to have
can we directly define
[test_idx] as below ?
[test_idx] SOURCE_KEY = MetaData:Sourcetype DEST_KEY = _MetaData:Index REGEX = test_sourcetype FORMAT = windows