Deployment Architecture

Is there a solution to back up Splunk data into HDFS to make it available for search via Hunk?

ddrillic
Ultra Champion

As a backup solution for Splunk’s data, we wonder what might be a solution to back up the Splunk data into HDFS and make it available for search via Hunk.

Any thoughts?

Tags (3)
0 Karma
1 Solution

rdagan_splunk
Splunk Employee
Splunk Employee

In addition to the above Hunk archiving recommendation, I would add the Hadoop Connect App exporting as another option: https://docs.splunk.com/Documentation/HadoopConnect/1.2.3/DeployHadoopConnect/ExporttoHDFS

View solution in original post

rdagan_splunk
Splunk Employee
Splunk Employee

In addition to the above Hunk archiving recommendation, I would add the Hadoop Connect App exporting as another option: https://docs.splunk.com/Documentation/HadoopConnect/1.2.3/DeployHadoopConnect/ExporttoHDFS

ddrillic
Ultra Champion

We wonder about the usage of shuttl - an open source software which is listed at -

shuttl

It says -

-- Shuttl works on the bucket level, and leverages the standard Splunk mechanism for archiving data based on total data size or time expiration.

What do you think about it?

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

I would not recommend you use Shuttl. It has not been maintained in over 3 years and was not tested on Splunk 6.* and would recommend you use Hunk Archiving or Hadoop Connect export.

0 Karma

ddrillic
Ultra Champion

Much appreciated. But even if it was supported, does moving the Splunk buckets, result in a Hunk "certified" underlying indexes?

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

Yes, both Hunk Archiving and Hadoop Connect App export are a certified solution.

0 Karma

splunkIT
Splunk Employee
Splunk Employee

ddrillic
Ultra Champion

Right, but we are looking for a backup solution not an archiving one...

0 Karma
Get Updates on the Splunk Community!

.conf25 Registration is OPEN!

Ready. Set. Splunk! Your favorite Splunk user event is back and better than ever. Get ready for more technical ...

Detecting Cross-Channel Fraud with Splunk

This article is the final installment in our three-part series exploring fraud detection techniques using ...

Splunk at Cisco Live 2025: Learning, Innovation, and a Little Bit of Mr. Brightside

Pack your bags (and maybe your dancing shoes)—Cisco Live is heading to San Diego, June 8–12, 2025, and Splunk ...