How to use a MapR cluster for frozen/archival storage in Splunk?

[edmtran]

Hello,

Let me first preface this by saying that I am very new to Splunk, MapR, NFS, and big data in general. I tried researching, but a lot of documentation / forum Answers go over my head or require adaptations (which I don't know how to do) to fit my scenario.

I have a Splunk cluster with 8 indexers, 3 search heads, and 2 admin nodes. Instead of using another server for frozen data storage, I would like to use a small MapR cluster so that I can create one volume per indexer. My instructions are to "create a NFS-exported directory and then create one dedicated directory per indexer in that NFS-exported directory."

I read this article, http://docs.splunk.com/Documentation/Splunk/6.4.3/Indexer/Automatearchiving, but am unsure whether I should go with automatic archiving or archiving with a script. These instructions to modify indexes.conf are still unclear to me:

[insert index here]
coldToFrozenDir = "insert path to frozen archive here"

What is an example of such a path in a MapR setting? How do I create a frozen archive? Isn't there a special way to load data into MapR or will Splunk take care of this?

Then I'm reading something about Hunk and Splunk Hadoop Connect which clouds up the picture. I don't think I will be using Hunk, but do I need Splunk Hadoop Connect to make Splunk work with MapR?

Thank you in advance for any guidance or advice you can provide.

[ddrillic]

It gets a bit tricky ; -)

Your case is perfect for a Splunk/Hunk combo because the moment you archive the Splunk data, the data will be transferred to Hadoop and Hunk would be the interface for the data.

Keep in mind that Hunk is Splunk with the exception that the data storage is in Hadoop and the data is not indexed to this Hunk instace (or cluster)

http://docs.splunk.com/Documentation/Splunk/6.4.3/Indexer/Automatearchiving

says -

-- You can configure the indexer to archive your data automatically as it ages; specifically, at the point when it rolls to "frozen". To do this, you configure indexes.conf.

Then it explains in detail more about it...

[edmtran]

Thanks for your response. As I said in my post, I did read that article, but would like an example to clarify the procedure. Because this is a MapR cluster and not just a storage server with an IP address, I am unsure of how to configure indexes.conf.