Getting Data In

Moving Splunk data to S3 bucket in a Cluster environment

shrogers
Loves-to-Learn Everything

Hi Everyone,

I'm looking for a working package that can move data from the Splunk cluster environment to the S3 bucket for archiving. All examples I'm getting does work.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Well, there's SmartStore, which is built-in to Splunk.

Beyond that, we'll need more information.  Is your Splunk on-prem, private cloud, or Splunk Cloud?  How do you want the data stored in S3 (searchable by Splunk or something else)?

What examples have you tried so far?

---
If this reply helps you, Karma would be appreciated.
0 Karma

shrogers
Loves-to-Learn Everything

Thank you your quick response. Smartstore would require a whole new setup and we are not able to go down that route.

It's an on-prem cluster environment. We just want to archive index data to S3 after 90 days. If we need to get it searchable, we'll get it done manually.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Write a coldToFrozenScript.  This script is invoked by Splunk when a bucket is due to be archived.  See 'coldToFrozenScript ' in the Admin manual (https://docs.splunk.com/Documentation/Splunk/8.1.1/Admin/Indexesconf#indexes.conf.spec)

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...

The Latest Cisco Integrations With Splunk Platform!

Join us for an exciting tech talk where we’ll explore the latest integrations in Cisco + Splunk! We’ve ...

AI Adoption Hub Launch | Curated Resources to Get Started with AI in Splunk

Hey Splunk Practitioners and AI Enthusiasts! It’s no secret (or surprise) that AI is at the forefront of ...