All Apps and Add-ons
Highlighted

Does Splunk and Elastic Map Reduce work Together?

SplunkTrust
SplunkTrust

I have a few indexes which have around 2.5 billion events each. Unfortunately we don't have a lot of CPU to sort through this massive data and make it meaningful in a dashboard. We're currently in the process of setting up a summary index, but the requirements/fields can change at anytime which mean's we'd have to re-summerize that data.

So my question is, can we use Amazon EMR as a temporary boost in horsepower to Map and Reduce this data back into the summary index? How difficult would this be to do?

Highlighted

Re: Does Splunk and Elastic Map Reduce work Together?

Path Finder

Hi,

I would assume your instance with 2,5 billion events is also running on AWS?

Why not use HUNK on AWS and export your data with the Splunk Hadoop Connect App?

https://aws.amazon.com/de/elasticmapreduce/hunk/

HTH,

Holger

View solution in original post

Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.