Hi all,
is there a limitation in the combination of transforms on a source in props.conf?
here is what i did and somehow I don't get any result.
Whenever I delete the TRANSFORMS-reroute entry, data is received and hostnames are changed.
Somehow I don't get the source with my regex rerouted to another index.
props.conf:
[source::tcp:514]
TRUNCATE = 64000
TRANSFORMS = newhost1
TRANSFORMS = newhost2
TRANSFORMS-reroute=set-index
transforms.conf
[newhost1]
DEST_KEY = MetaData:Host
REGEX = mymatchinghost1rex
FORMAT = host::myhost1
[newhost2]
DEST_KEY = MetaData:Host
REGEX = mymatchinghost2rex
FORMAT = host::myhost2
[set-index]
DEST_KEY=_MetaData:Index
REGEX= .+mymatchingrex.+
FORMAT=myindex
WRITE_META=true
thanks for your help,
kind regards,
harald
I'm not aware of a limitation and, if there is one, it is probably a very large number.
As per the spec, all the props.conf entries should use the TRANSFORMS-<class> notation so maybe that is an issue - it kinda works when the set-index entry is removed by luck.
This should work
[source::tcp:514]
TRUNCATE = 64000
TRANSFORMS-hostrename = newhost1, newhost2
TRANSFORMS-reroute = set-index
For the transforms.conf set-index entry I do not think the WRITE_META parameter is needed as it's not a index-time field extraction
WRITE_META = <boolean> * NOTE: This setting is only valid for index-time field extractions. * Automatically writes REGEX to metadata. * Required for all index-time field extractions except for those where DEST_KEY = _meta (see the description of the DEST_KEY setting, below) * Use instead of DEST_KEY = _meta. * Default: false
The .+ is not really necessary at the start/end of the REGEX either, but it should also do no harm. Anyway, try this and see if it helps.
[set-index]
DEST_KEY = _MetaData:Index
REGEX = mymatchingrex
FORMAT = myindex
#WRITE_META = true
Restart the test heavy forwarder/indexer to ensure the configs are applied.
Hope this helps.
I'm not aware of a limitation and, if there is one, it is probably a very large number.
As per the spec, all the props.conf entries should use the TRANSFORMS-<class> notation so maybe that is an issue - it kinda works when the set-index entry is removed by luck.
This should work
[source::tcp:514]
TRUNCATE = 64000
TRANSFORMS-hostrename = newhost1, newhost2
TRANSFORMS-reroute = set-index
For the transforms.conf set-index entry I do not think the WRITE_META parameter is needed as it's not a index-time field extraction
WRITE_META = <boolean> * NOTE: This setting is only valid for index-time field extractions. * Automatically writes REGEX to metadata. * Required for all index-time field extractions except for those where DEST_KEY = _meta (see the description of the DEST_KEY setting, below) * Use instead of DEST_KEY = _meta. * Default: false
The .+ is not really necessary at the start/end of the REGEX either, but it should also do no harm. Anyway, try this and see if it helps.
[set-index]
DEST_KEY = _MetaData:Index
REGEX = mymatchingrex
FORMAT = myindex
#WRITE_META = true
Restart the test heavy forwarder/indexer to ensure the configs are applied.
Hope this helps.
Hi @yeahnah,
That did the trick.
It's working now...
hosts are renamed and data is rerouted into the correct index.
Thanks a lot,
cheers