How to write transforms.conf for the fields that are not present in metadata
For example, I need to write transforms for the field - asset_env
asset_env = PROD
Below transforms were not working.
SOURCE_KEY = field:asset_env
REGEX = ^asset_env::(\w+)
DEST_KEY = _MetaData:Index
FORMAT = index_$1
let me understand: you want to create an index based on a condition (e.g. if host=host_prod, env=PROD otherwise is env=DEV), is it correct?
in this case you can create a calculated field [Settings -- Fields -- Calculated fields -- New calculated field] adding the rule for your sourcetype, e.g. for the above example
Host = abcdpr,xyzpr,abcps,xyzps etc......
asset_env = PROD,PSUP,PSVT etc...
We have different different asset_env like asset_env = PROD, asset_env=PSVT, asset_env=PSUP likewise.
I will create index like (index_PROD, index_PSUP, index_PSVT)
Based on the transforms.conf, the logs based on asset_env should go to respective index
The easiest way is to manage the correct index addressingusing diferent inputs.conf for each host (you can manage them using a Deployment Server).
Otherwise you could override index on Indexers (of on Heavy Forwarders if present) based on the host value, something like this:
[host::host_PROD] TRANSFORMS-override_index_PROD = override_index_PROD [host::host_DEV] TRANSFORMS-override_index_DEV = override_index_DEV
[override_index_PROD] REGEX = . DEST_KEY = _MetaData:Index FORMAT = index_PROD [override_index_DEV] REGEX = . DEST_KEY = _MetaData:Index FORMAT = index_DEV