I understand Splunk Hadoop Connect is a free app and Hunk License depends on the no of Tasktrackers.
We have Splunk Enterprise in our organisation and the goal is to perform analytics on Hadoop data and send archived data to Hadoop from Indexes.
I can achieve this via both Splunk Hadoop Connect and Hunk, but my doubt is what's the difference between these two w.r.t licensing, other than the bidirectional data movement that Hadoop Connect provides?
Now if I get Splunk Hadoop Connect app, then the licensing will depend on what parameters?
Once data has been indexed in Splunk, there is no additional cost for copying it to Hadoop. Both Splunk Hadoop Data Roll (used to be called Hunk Archiving) and Splunk Hadoop Connect Exporting do not charge for coping the data to Hadoop.
so does it cost when I use Splunk Hadoop connect for Hadoop Data Analytics?
What exactly is the parameter on which Splunk Hadoop Connect licensing is based?
Hadoop Connect has three options: Export to Hadoop, Import from Hadoop, and exploring Hadoop data using search commands.
The Export to Hadoop sends data from Splunk Search to HDFS and does not have any additional license to the Splunk license.
The Import from Hadoop sends data from HDFS to Splunk Indexers, so that will have an impact on your Splunk license.
The exploring Hadoop data does not have any additional license to the Splunk license
Oh..!! So as per my knowledge, Splunk enterprise comes along with Hunk/Splunk analytics for Hadoop.
Extra licensing is incurred only if we import data from Hadoop to Splunk.
We have licensed splunk Enterprise 6.3.2 in our organization, Why is that I cant see "Virtual Indexes" in Splunk Search head UI ??
I don't wanna import data from Hadoop.I wanna use to view/analyse Hadoop data.
So with my current license, without incurring additional cost I can analyse Hadoop Data right? But I don't see option for "Virtual Indexes" in our Splunk Web SH UI.
Please correct me if I'm wrong.
Hunk (before 6.5) or Splunk Analytics for Hadoop (since 6.5), requires a license. It is licensed by the number of Hadoop Nodes and not by the amount of data indexed.
Therefore, in version 6.3.2 you will need a Hunk license to see the Virtual Index option.
hI @rdagan ,
Sorry to bug you again.
So my understanding is that - " Hunk/Splunk Analytics for Hadoop " requires additional license for analysing Hadoop data ( Hunk license depends on no. of Hadoop nodes) .
1.My goal is to analyse Hadoop data and neither import or export data..
My understanding is that Hadoop connect just explores the files/directories using search commands and cannot be used for analysis , reporting etc unless the data is imported from Hadoop to Splunk Indexers.
Where as Hunk can perform this without indexing the data via Virtual Indexes?
Am I right?
2.My MAIN doubt is When they say Hadoop Connect can "Explore and browse HDFS directories and files" --> what all features can I achieve via this? Just view the contents of HDFS files, is it ? Is searching restricted here and in what way?
I can use stats,chart,timechart..commands for hadoop data using a Hadoop connect ?
1) You are correct. Splunk Analytics for Hadoop can do the analysis without indexing the data. As you highlight it is done by using Virtual Indexing. Yes, Splunk Analytics for Hadoop is licensed by the number of Hadoop nodes.
2) This link shows some of the commands you can use with Splunk Hadoop Connect to explore the data: http://blogs.splunk.com/2012/12/20/connecting-splunk-and-hadoop/
For example | hdfs read file.txt OR | hdfs lsr /directory
@sarnagar - Did the answer provided by rdagan help provide a solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!