topic Re: How do hunk/hadoop searches work? in Splunk Search

How do hunk/hadoop searches work?

JohnTelus — Wed, 26 Nov 2014 16:39:37 GMT

Does, for example, hunk retrieve all the data from the hadoop path, move it to a temporary location, apply the search criteria, and then return the results to the user?

Re: How do hunk/hadoop searches work?

rdagan_splunk — Wed, 26 Nov 2014 17:26:34 GMT

Hunk converts the search into Hadoop MR Job. With virtual indexes, Hunk can access subset of the data. Hunk leverages the MapReduce framework to execute report-generating searches and Indexing on Hadoop nodes. Data does not need to be pre-processed before it is accessed because Hunk lets you run analytics searches against the data where it rests in Hadoop. In addition, Data Preview for Exploration is done by allowing Hunk to look at subset of the data after each phase of the MR Job.

Re: How do hunk/hadoop searches work?

Ledion_Bitincka — Wed, 26 Nov 2014 18:40:08 GMT

I'd highly recommend these three blog posts (and obviously our docs) for more info:

http://blogs.splunk.com/2013/06/26/hunk-splunk-analytics-for-hadoop-intro-%E2%80%93-part-1/
http://blogs.splunk.com/2013/07/07/hunk-intro-part-2/
http://blogs.splunk.com/2013/11/08/hunk-intro-part-3/