Export data to Hadoop

[siva_cg]

Hi Team,

I want to export data (not archiving data) from Splunk running on version 7.1.3 to Hadoop server in two cases.

1st case - Set of data using Splunk search
2nd case - All the events in a particular sourcetype from an index (I can create a separate index for that specific sourcetype and send whole index data to Hadoop)

In both cases, Can I use Hadoop Connect app to export data? I read that from Splunk version 6.5, we need to use Splunk Analytics for Hadoop but didn't find any document that says about data export using Splunk Analytics for Hadoop. Could you please suggest which app to use? Thanks in advance.

[rdagan_splunk · Accepted solution]

Regarding your question about Splunk Hadoop Connect support Splunk 7.1.3 version - Splunk Hadoop Connect works without a problem with that version. And my recommendation is for you to ask your Splunk sales team to send that request to Splunk support.

[siva_cg]

Thank you @rdagan. Sure, I will contact my Splunk sales team for the request.

[inventsekar]

Hi Siva, i am not sure.. but did you check about "Hadoop Data Roll"

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/ArchivingindexestoHadoop

[siva_cg]

Hi @inventsekar,
As per my understand, "Hadoop Data Roll" is used to move data from Splunk to Hadoop and use hdfs search for any future reports or searches. But in my usecase, I just want to copy some data to Hadoop server.

[rdagan_splunk]

Yes, for both use cases Splunk Hadoop Connect is the right App.
Here is the link to the doc that will help you schedule that search and export:
http://docs.splunk.com/Documentation/HadoopConnect/latest/DeployHadoopConnect/Appdashboard

Just be aware that Splunk will normally recommend a dedicated Search Head if you plan on moving a large amount of data per day.

[siva_cg]

Hi @rdagan,

Thank you for your response. Yeah, we will use dedicated Search Head for this purpose. Will Splunk Hadoop Connect support Splunk 7.1.3 version ?