Archive

Can we export indexed data from Splunk to Hadoop via SplunkHadoopConnect?

Communicator

Are we able to export indexed data from Splunk to Hadoop without running searches via Splunk Hadoop Connect?

I know we could use Hunk for the same. But how far we could utilise Hadoop Connect in our environment to export the indexed data directly?

Thanks in advance.

Tags (2)
0 Karma

Splunk Employee
Splunk Employee

Well,

old question but worth to answer... introduced with V6.5 there is now the "old" Hadoop archiving feature available in Splunk Enterprise core without the need for an additional "Splunk Analytics for Hadoop" license.

The feature is called "Hadoop Data Roll".

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/ArchivingindexestoHadoop

No need for Hadoop Connect anymore if you want to archive buckets instead of search results (which are different use cases, obviously 🙂

HTH,

Holger

0 Karma

Splunk Employee
Splunk Employee

I'm not sure what you mean by directly here? Hadoop Connect is designed to allow the export of search results. Hunk's archiving is designed to allow archiving. What are you trying to do?

0 Karma

Communicator

Thanks Dart. We have not touched both practically. I have just gone through the docs and came to know this. Our requirement is to get archive the indexed data from Splunk to Hadoop. What i am trying to do is can we export the indexed data (buckets) without exporting the search results through any other means without Hunk? Is this possible in Hadoop Connect?

0 Karma

Splunk Employee
Splunk Employee

You cannot archive buckets using Hadoop Connect, but you can export events as documented here

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!