Is it possible to use a Universal Forwarder to write logs directly to HDFS?

[alessio23]

Hi,

I installed and configured Hunk to read data from HDFS.

I'm trying to use Universal Forwarder to write directly logs to Hdfs. Is it possible ? Is there any "component" of Universal Forwarder to do this ?

Many Thanks
Alessio

[Ledion_Bitincka]

Currently Splunk does not provide a way to write data directly to HDFS. Splunk forwarders can currently send data to either a Splunk indexer or a syslog receiver, however we're planning to expand that list. If you don't mind commenting what type of data are you trying to collect and what functionality would you want in the "collector" (parse, partition, filter, etc) ?

[suarezry]

One solution is to configure a data collector that writes to HDFS. You can then configure the Universal Forwarder to forward to the data collector. We use Fluentd (www.fluentd.org) as the data collector for it's ease of use and it's ability to collect data from many different sources.