HUNK vs Splunk Hadoop Connect: How to move data from Splunk to Hadoop and search the data in Hadoop?

[kml_uvce]

my requirement is to move data from splunk to hadoop and also search the data in hadoop

[ChrisG]

Hunk is for searching and analyzing data stored in Hadoop. Hadoop Connect enables you to move data in both directions between Splunk Enterprise and Hadoop. It's not an either/or. For your requirements, it's both.

See these two documentation topics for the overview:
- About Splunk Hadoop Connect
- Meet Hunk

[kml_uvce]

I think , I can also move data from splunk to hadoop by archiving old data in indexes and send it to hadoop

but in hadoop connect I can send aggregated search result data in Hadoop If I want but in HUNK I can not do this , I am not sure about it..
Also to read data from Hadoop to splunk ,HUNK does not require to index data in splunk but in hadoop connect we need to index data from Hadoop before reading ?

I heard HUNK is not stable , is this true ? please share your thoughts on this..

[ChrisG]

rdagan answered your archiving question, my apologies for leaving out that detail!

Hunk is stable and reliable, what specifically did you hear to the contrary?

[kml_uvce]

which one is better in terms of easy to use ?
Also to read/search data from Hadoop to splunk ,HUNK does not require to index data in splunk but in hadoop connect we need to first index data from Hadoop before reading/searching ?

[ChrisG]

Hunk uses virtual indexes to enable searching of Hadoop data.

Hadoop Connect imports the data into Splunk Enterprise, where it gets indexed like other data. Hadoop Connect provides an exploration feature so you can preview the files you want to index in Splunk Enterprise. You can also use the HDFS Explore feature to read results of a MapReduce job and display it alongside Splunk search results. If you use HDFS Explore in this way, the data is not indexed, but the feature is much more limited compared with regular Splunk search capabilities that are available in Hunk.

[kml_uvce]

Thanks Chris
Can you please give me some scenario/use cases where we should use HUNK and where we should use Splunk Hadoop Connect ?

[ChrisG]

If you have not already done so, I strongly suggest you look at the materials available on the Hunk product page on splunk.com, including the white paper and the customer stories. The Hadoop Connect use cases are pretty simple and are well-described on the Hadoop Connect app page.

If you have more detailed questions about the use cases for these products, you should contact Splunk Sales for more information.

[rdagan_splunk]

After the data was indexed in Splunk, you can use the below options to move data to HDFS
1) Hadoop Connect Export. This is the output of a Search and is done from the Search Head to HDFS
2) Hunk Archiving. For this option we copy the raw data directly from the Indexer (journal.gz file) to HDFS