Archive
Highlighted

Export data to Hadoop

Explorer

Hi Team,

I want to export data (not archiving data) from Splunk running on version 7.1.3 to Hadoop server in two cases.

1st case - Set of data using Splunk search
2nd case - All the events in a particular sourcetype from an index (I can create a separate index for that specific sourcetype and send whole index data to Hadoop)

In both cases, Can I use Hadoop Connect app to export data? I read that from Splunk version 6.5, we need to use Splunk Analytics for Hadoop but didn't find any document that says about data export using Splunk Analytics for Hadoop. Could you please suggest which app to use? Thanks in advance.

0 Karma
Highlighted

Re: Export data to Hadoop

Champion

Hi Siva, i am not sure.. but did you check about "Hadoop Data Roll"

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/ArchivingindexestoHadoop

0 Karma
Highlighted

Re: Export data to Hadoop

Explorer

Hi @inventsekar,
As per my understand, "Hadoop Data Roll" is used to move data from Splunk to Hadoop and use hdfs search for any future reports or searches. But in my usecase, I just want to copy some data to Hadoop server.

0 Karma
Highlighted

Re: Export data to Hadoop

Splunk Employee
Splunk Employee

Yes, for both use cases Splunk Hadoop Connect is the right App.
Here is the link to the doc that will help you schedule that search and export:
http://docs.splunk.com/Documentation/HadoopConnect/latest/DeployHadoopConnect/Appdashboard

Just be aware that Splunk will normally recommend a dedicated Search Head if you plan on moving a large amount of data per day.

0 Karma
Highlighted

Re: Export data to Hadoop

Explorer

Hi @rdagan,

Thank you for your response. Yeah, we will use dedicated Search Head for this purpose. Will Splunk Hadoop Connect support Splunk 7.1.3 version ?

0 Karma
Highlighted

Re: Export data to Hadoop

Splunk Employee
Splunk Employee

Regarding your question about Splunk Hadoop Connect support Splunk 7.1.3 version - Splunk Hadoop Connect works without a problem with that version. And my recommendation is for you to ask your Splunk sales team to send that request to Splunk support.

View solution in original post

0 Karma
Highlighted

Re: Export data to Hadoop

Explorer

Thank you @rdagan. Sure, I will contact my Splunk sales team for the request.

0 Karma