How to use shuttl to archive frozen data in local FS

[alanwill]

I'm trying to implement shuttl in our day to day Splunk workflow and in setting it up, it looks like it would shuttl my cold data to the new frozen location, say S3. However I already have many terabytes of frozen data stored locally that I'd like to shuttl to S3, can I still use shuttl for this?

The idea would be to copy the existing local mounted SAN frozen archives to S3, then use the coldToFrozenScript from then on to move directly to S3.

How can I do this? Thanks,
alan

[Petter_Eriksson · Accepted solution]

You could setup Shuttl and call its copy/freeze scripts manually.

Here's how you do it:
1. Install and configure Shuttl.
2. Start Splunk and make sure Shuttl is running by looking at the Shuttl dashboard.
3. call $SPLUNK_HOME/etc/apps/shuttl/bin/warmToColdScript.sh /absolute/path/to/bucket /absolute/path/to/bucket
And your buckets should now be copied.

And yes, I've written "/absolute/path/to/bucket" twice. That's because usually when you use that script, you want to move the bucket from Splunk's warm to cold directory.

Alternative: Use this modified script, placed in Shuttl's bin directory: https://gist.github.com/petterik/7793825
Call it with a single argument of the bucket's absolute path. Installing, configuring and starting Shuttl is still needed.