Getting Data In

Where is the doc for the 6.5 hadoop data roll?

a212830
Champion

Hi,

I'm searching for the documentation for the new 6.5 hadoop data roll feature, and unable to find it. Can someone point me to it? Or where it's setup within Splunk? Nothing obvious stands out.

0 Karma
1 Solution

inventsekar
Super Champion

About archiving indexes with Hadoop Data Roll
http://docs.splunk.com/Documentation/Splunk/6.5.0/Indexer/ArchivingindexestoHadoop

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.

View solution in original post

inventsekar
Super Champion

About archiving indexes with Hadoop Data Roll
http://docs.splunk.com/Documentation/Splunk/6.5.0/Indexer/ArchivingindexestoHadoop

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.

ChrisG
Splunk Employee
Splunk Employee

Searching on docs.splunk.com for hadoop data roll should turn this topic right up.

0 Karma

a212830
Champion

Thanks. I'll do that from now on - Mr. Google didn't find it.

0 Karma

a212830
Champion

Thanks. Couldn't find that via Mr. Google....

So, next stupid question - the doc doesn't indicate that Hunk is required to search this data after it's archived. Is that accurate? I can query my data in hadoop without requiring Hunk?

0 Karma

inventsekar
Super Champion

i think that is accurate.

You can search archived buckets as you normally search, simply include the archive virtual index in your searches. See Search archived index data (http://docs.splunk.com/Documentation/Splunk/6.5.0/Indexer/Archivesearchtips) for information about search commands that work with indexes stored in Hadoop.

You can for example, create one search that searches Splunk for:

Data in a Splunk Enterprise index.
Archived data copied into HDFS or S3.

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma

a212830
Champion

Holy moly! So.......... next question - just data that was once in Splunk, or any data that is now in Hadoop?

0 Karma

inventsekar
Super Champion

per my understanding, just the data that was once in splunk, now archived into HDFS/S3.

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma

hsesterhenn_spl
Splunk Employee
Splunk Employee

Yeah... you have paid for the data to be indexed in Splunk Enterprise...
You don't have to pay for the archived data again.

If you ingest data directly into HDFS (using Flume e.g.) you haven't paid in Splunk land... you'll need a license for Splunk Analytics for Hadoop, formerly known as HUNK :-).

Does it make sense?

0 Karma

a212830
Champion

Figured. (That would have been too good to be true).

Still, it's a big help!

0 Karma

inventsekar
Super Champion

thanks, can you please accept this as the answer..

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...