topic Re: How can I merge _meta from several inputs.conf files in Getting Data In

How can I merge _meta from several inputs.conf files

cwacha — Mon, 06 Jun 2011 09:47:11 GMT

I use the universal forwarders ability to enrich the transported files with _meta keywords as follows:

./etc/apps/myapp/local/inputs.conf

[monitor:///myfile]
  disabled = false
  _meta = key1::value key2::value

I also have global key/value pairs for _meta that I would like to add automatically to all monitor stanzas. They are defined in

./system/local/inputs.conf

[default]
  _meta = globalkey::value

The globalkey keyword gets added to all monitor stanzas that do not define a specific _meta keyword. I would like to have the globalkey keyword as well as the additionally defined key/value pairs in the apps inputs.conf defined. Unfortunately the _meta field in apps/../inputs.conf overwrites the system/local/inputs.conf _meta entry.

Is it possible to append the global keywords (defined in ./system/local/) to the defined _meta tag (defined in ./apps/local/inputs.conf) ?.

For example with a configuration as follows using $_meta:

./system/local/inputs.conf

[default]
  _meta = globalkey::value

./etc/apps/myapp/local/inputs.conf

[monitor:///myfile]
  disabled = false
  _meta = $_meta key1::value key2::value

Re: How can I merge _meta from several inputs.conf files

jbsplunk — Mon, 06 Jun 2011 15:29:42 GMT

I don't think it is possible to merge entries from inputs.conf in the manner your describing here. The way precedence works is to take the stanza and do merging based on the settings. The setting with the highest priority is what is taken into account. Other settings will be ignored.

You can probably do this with a props/transforms configuration on the stanzas where you want this to occur.

Re: How can I merge _meta from several inputs.conf files

cwacha — Tue, 07 Jun 2011 08:19:32 GMT

As fas as I know props/transforms cannot be used with the universal forwarder...

Re: How can I merge _meta from several inputs.conf files

jbsplunk — Tue, 07 Jun 2011 14:56:54 GMT

The universal forwarder can't do much in the way of parsing, but you can do it at the indexer that the UF is reporting into without any problem.