Getting Data In

Anyone splunking hadoop logs?

pde
Path Finder

I'd be interested in talking about saved searches, etc around hadoop logs. Anybody got a head start?

Thanks!

pierre4splunk
Splunk Employee
Splunk Employee

The Splunk App for HadoopOps is now generally available.

http://splunk-base.splunk.com/apps/57004/splunk-app-for-hadoopops

The App is free. You can learn more here:

http://www.splunk.com/view/hadoopops/SP-CAAAHA2

pierre4splunk
Splunk Employee
Splunk Employee

here at splunk R&D, we've been splunking Hadoop logs for awhile: daemon logs, counters and measures... metrics, metrics, metrics. things get very interesting when you're able to correlate this with system info for each node in the cluster, client activity, configuration settings, and more. we've developed an app that collects all this for splunk search, along with rich dashboards and interactive UIs designed for both Hadoop operators and developers' needs.

interested in trying it out? we'd love to share ideas and get your feedback.

To be eligible, register for the splunk enterprise for Hadoop beta program on splunkbase. Be sure to describe what you're interested in too -- i.e. mention 'monitor' or 'splunking Hadoop logs' in the web form.

0 Karma

Damien_Dallimor
Ultra Champion

Have a look at Splunk for JMX... Hadoop has loads of Mbeans that you can monitor across your cluster(s).
Many folks are already using this Splunk app for Hadoop and Cassandra monitoring.

Damien.

0 Karma

smcavoyams
New Member

Could anyone post saved searches useful field extraction regexs, etc. for hadoop/hbase/etc. ?

0 Karma

Archana
Splunk Employee
Splunk Employee

I did some performance modeling based on data from hadoop logs and have recently gotten up to speed writing saved searches in Splunk. I'd be happy to help you put the two together.

0 Karma

pde
Path Finder

For monitoring & troubleshooting mapreduce jobs and for monitoring/troubleshooting of hadoop clusters overall. My thinking is around a Hadoop app that would compliment and augment hadoop/ganglia.

0 Karma

Dan
Splunk Employee
Splunk Employee

Is this for monitoring and troubleshooting Hadoop jobs?

0 Karma
Get Updates on the Splunk Community!

Happy CX Day, Splunk Community!

Happy CX Day, Splunk Community! CX stands for Customer Experience, and today, October 3rd, is CX Day — a ...

Splunk Observability Cloud | Customer Survey!

If you use Splunk Observability Cloud, we invite you to share your valuable insights with us through a brief ...

.conf23 | Get Your Cybersecurity Defense Analyst Certification in Vegas

We’re excited to announce a new Splunk certification exam being released at .conf23! If you’re going to Las ...