Getting Data In

Anyone splunking hadoop logs?

pde
Path Finder

I'd be interested in talking about saved searches, etc around hadoop logs. Anybody got a head start?

Thanks!

pierre4splunk
Splunk Employee
Splunk Employee

The Splunk App for HadoopOps is now generally available.

http://splunk-base.splunk.com/apps/57004/splunk-app-for-hadoopops

The App is free. You can learn more here:

http://www.splunk.com/view/hadoopops/SP-CAAAHA2

pierre4splunk
Splunk Employee
Splunk Employee

here at splunk R&D, we've been splunking Hadoop logs for awhile: daemon logs, counters and measures... metrics, metrics, metrics. things get very interesting when you're able to correlate this with system info for each node in the cluster, client activity, configuration settings, and more. we've developed an app that collects all this for splunk search, along with rich dashboards and interactive UIs designed for both Hadoop operators and developers' needs.

interested in trying it out? we'd love to share ideas and get your feedback.

To be eligible, register for the splunk enterprise for Hadoop beta program on splunkbase. Be sure to describe what you're interested in too -- i.e. mention 'monitor' or 'splunking Hadoop logs' in the web form.

0 Karma

Damien_Dallimor
Ultra Champion

Have a look at Splunk for JMX... Hadoop has loads of Mbeans that you can monitor across your cluster(s).
Many folks are already using this Splunk app for Hadoop and Cassandra monitoring.

Damien.

0 Karma

smcavoyams
New Member

Could anyone post saved searches useful field extraction regexs, etc. for hadoop/hbase/etc. ?

0 Karma

Archana
Splunk Employee
Splunk Employee

I did some performance modeling based on data from hadoop logs and have recently gotten up to speed writing saved searches in Splunk. I'd be happy to help you put the two together.

0 Karma

pde
Path Finder

For monitoring & troubleshooting mapreduce jobs and for monitoring/troubleshooting of hadoop clusters overall. My thinking is around a Hadoop app that would compliment and augment hadoop/ganglia.

0 Karma

Dan
Splunk Employee
Splunk Employee

Is this for monitoring and troubleshooting Hadoop jobs?

0 Karma
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...