Getting Data In

Moving Splunk data to S3 bucket in a Cluster environment

shrogers
Loves-to-Learn

Hi Everyone,

I'm looking for a working package that can move data from the Splunk cluster environment to the S3 bucket for archiving. All examples I'm getting does work.

Labels (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Well, there's SmartStore, which is built-in to Splunk.

Beyond that, we'll need more information.  Is your Splunk on-prem, private cloud, or Splunk Cloud?  How do you want the data stored in S3 (searchable by Splunk or something else)?

What examples have you tried so far?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

shrogers
Loves-to-Learn

Thank you your quick response. Smartstore would require a whole new setup and we are not able to go down that route.

It's an on-prem cluster environment. We just want to archive index data to S3 after 90 days. If we need to get it searchable, we'll get it done manually.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Write a coldToFrozenScript.  This script is invoked by Splunk when a bucket is due to be archived.  See 'coldToFrozenScript ' in the Admin manual (https://docs.splunk.com/Documentation/Splunk/8.1.1/Admin/Indexesconf#indexes.conf.spec)

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.