I am trying to pick up to files in specific directories under different sourectypes.
[monitor:///app/ems-store-uat/uat/.../config/queues.conf] sourcetype = ems_queues disabled = false [monitor:///app/ems-store-uat/uat/.../config/topics.conf] sourcetype = ems_topics disabled = false
The files exist in multiple paths such as /app/ems-store-uat/uat/U1_LN_DERIV_TEST/config/queues.conf & /app/ems-store-uat-uat/U1_LN_DERIV_TEST/config/topics.conf.
I want them under separate sourcetypes, because I want to group them by different type of config, but it seems that the first one is blocking the second one - the topics.conf get blacklisted, perhaps by the first?
04-19-2010 10:43:09.212 INFO TailingProcessor - Adding /app/ems-store-uat/uat/U1_LN_DERIV_STAGING_DESFOCASH/config/topics.conf to ignore list. 04-19-2010 10:43:09.492 DEBUG TailingProcessor - Ignoring non-whitelisted file: /app/ems-store-uat/uat/U1_LN_DERIV_AIRLOCK/config/topics.conf 04-19-2010 10:43:09.492 INFO TailingProcessor - Adding /app/ems-store-uat/uat/U1_LN_DERIV_AIRLOCK/config/topics.conf to ignore list.
Is there a way that I can do this?
The behavior you're describing sounds like a bug. You've specified a whitelist by naming the log file in your monitor input. Please file a support ticket.
In the meantime, you should be able to use a single monitor input in conjunction with props.conf to get this to work:
inputs.conf: [monitor:///app/ems-store-uat/uat/.../config] _whitelist = (topics\.conf|queues\.conf)$ props.conf: [source::.../topics.conf] sourcetype=ems_topics [source::.../queues.conf] sourcetype=ems_queues
should also whitelist (?:topics.conf|queues.conf)$ if there might be other files in the directory you don't want.
Please let us know the version of your forwarder/monitor, as there were significant changes made as of 4.1.
4.1 will work the way you have configured above, but 4.0 and below will require tina_p's method below to work reliably.