Deployment Architecture

Is there a solution to back up Splunk data into HDFS to make it available for search via Hunk?

ddrillic
Ultra Champion

As a backup solution for Splunk’s data, we wonder what might be a solution to back up the Splunk data into HDFS and make it available for search via Hunk.

Any thoughts?

Tags (3)
0 Karma
1 Solution

rdagan_splunk
Splunk Employee
Splunk Employee

In addition to the above Hunk archiving recommendation, I would add the Hadoop Connect App exporting as another option: https://docs.splunk.com/Documentation/HadoopConnect/1.2.3/DeployHadoopConnect/ExporttoHDFS

View solution in original post

rdagan_splunk
Splunk Employee
Splunk Employee

In addition to the above Hunk archiving recommendation, I would add the Hadoop Connect App exporting as another option: https://docs.splunk.com/Documentation/HadoopConnect/1.2.3/DeployHadoopConnect/ExporttoHDFS

ddrillic
Ultra Champion

We wonder about the usage of shuttl - an open source software which is listed at -

shuttl

It says -

-- Shuttl works on the bucket level, and leverages the standard Splunk mechanism for archiving data based on total data size or time expiration.

What do you think about it?

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

I would not recommend you use Shuttl. It has not been maintained in over 3 years and was not tested on Splunk 6.* and would recommend you use Hunk Archiving or Hadoop Connect export.

0 Karma

ddrillic
Ultra Champion

Much appreciated. But even if it was supported, does moving the Splunk buckets, result in a Hunk "certified" underlying indexes?

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

Yes, both Hunk Archiving and Hadoop Connect App export are a certified solution.

0 Karma

splunkIT
Splunk Employee
Splunk Employee

ddrillic
Ultra Champion

Right, but we are looking for a backup solution not an archiving one...

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...