I am trying to override the host field based on an element in the source path. This is data that is being forwarded from a universal forwarder and we want the host the data originally came from as the host field, not the hostname of the forwarder.
To date, I have tried modifying etc/system/local/transforms.conf and props.conf both on the indexers and in the search app on our search heads; all with no luck. I have also tried various values in transforms.conf ranging from both MetaData:Source and source for SOURCE_KEY and using source:: in the regex. Anyone have any ideas what I'm doing wrong?
Here is what is currently in transforms.conf and props.conf:
transforms.conf:
[set-host-elasticsearch]
SOURCE_KEY = MetaData:Source
REGEX = /hosts/([^/]+)/logs/
DEST_KEY = MetaData:Host
FORMAT = host::$1
props.conf:
[elasticsearch]
TRANSFORMS-set-host-elasticsearch = set-host-elasticsearch
The regex works fine in a search, I can run the following and get a table of hosts:
sourcetype=elasticsearch | rex field=source "/hosts/(?<hostname>[^/]+)/logs/" | stats count by hostname
As an example, my source path might be something like:
/rel/ps/applications/elasticsearch/hosts/tvrap571/logs/test_cre_gld.log
... View more