The Splunk App for HadoopOps is now generally available.
The App is free. You can learn more here:
here at splunk R&D, we've been splunking Hadoop logs for awhile: daemon logs, counters and measures... metrics, metrics, metrics. things get very interesting when you're able to correlate this with system info for each node in the cluster, client activity, configuration settings, and more. we've developed an app that collects all this for splunk search, along with rich dashboards and interactive UIs designed for both Hadoop operators and developers' needs.
interested in trying it out? we'd love to share ideas and get your feedback.
To be eligible, register for the splunk enterprise for Hadoop beta program on splunkbase. Be sure to describe what you're interested in too -- i.e. mention 'monitor' or 'splunking Hadoop logs' in the web form.
Have a look at Splunk for JMX... Hadoop has loads of Mbeans that you can monitor across your cluster(s).
Many folks are already using this Splunk app for Hadoop and Cassandra monitoring.
For monitoring & troubleshooting mapreduce jobs and for monitoring/troubleshooting of hadoop clusters overall. My thinking is around a Hadoop app that would compliment and augment hadoop/ganglia.