Is there a solution to back up Splunk data into HDFS to make it available for search via Hunk?

[ddrillic]

As a backup solution for Splunk’s data, we wonder what might be a solution to back up the Splunk data into HDFS and make it available for search via Hunk.

Any thoughts?

[rdagan_splunk · Accepted solution]

In addition to the above Hunk archiving recommendation, I would add the Hadoop Connect App exporting as another option: https://docs.splunk.com/Documentation/HadoopConnect/1.2.3/DeployHadoopConnect/ExporttoHDFS

[ddrillic]

We wonder about the usage of shuttl - an open source software which is listed at -

shuttl

It says -

-- Shuttl works on the bucket level, and leverages the standard Splunk mechanism for archiving data based on total data size or time expiration.

What do you think about it?

[rdagan_splunk]

I would not recommend you use Shuttl. It has not been maintained in over 3 years and was not tested on Splunk 6.* and would recommend you use Hunk Archiving or Hadoop Connect export.

[ddrillic]

Much appreciated. But even if it was supported, does moving the Splunk buckets, result in a Hunk "certified" underlying indexes?

[rdagan_splunk]

Yes, both Hunk Archiving and Hadoop Connect App export are a certified solution.

[splunkIT]

Please take a look at this blog:
http://blogs.splunk.com/2015/01/21/new-in-hunk-6-2-1-splunk-archiving-searchable-archives/

[ddrillic]

Right, but we are looking for a backup solution not an archiving one...