All Apps and Add-ons
Highlighted

Help on HadoopOps

Communicator

Hi all,
Anybody aware of these warnings in HadoopOps app?

Lookup table 'hadoophost2maxcpu' is empty.
No matching fields exist
Lookup table 'hadoop
host2mapred' is empty.

why am getting these?
how to resolve?
Should i need to do any conf setting??

Thanks in advance.

Highlighted

Re: Help on HadoopOps

Communicator

IF ANY ONE KNOWS PLZ HELP ME

0 Karma
Highlighted

Re: Help on HadoopOps

Splunk Employee
Splunk Employee

This suggest that 'ps' data are not getting in from forwarder (Splunk_TA_hadoopops). Is [script://./bin/hadoopmon_ps.sh] enabled?

Edit:
As pierre4splunk replied below, core-site.xml and mapred-site.xml need to be monitored to create hdfs and mapred lookup, respectively. maxcpu lookup depends on results from hadoopmon_cpu.sh. Might be worth double-checking.

View solution in original post

Highlighted

Re: Help on HadoopOps

Splunk Employee
Splunk Employee

These lookups are populated when the app is first initialized, and then updated once per day thereafter. Are you seeing this after a new deployment or upgrade? If so, this message could show up if some data is not yet available. The message persists until the next scheduled search updates the lookup correctly.

You can run these scheduled searches manually, both to troubleshoot and to populate the lookup directly. I've pasted them here for reference (expanded from saved searches and macros):

index=hadoopmon_os source="cpu" | stats max(CPU) as maxcpu by host | eval maxcpu = maxcpu + 1 | outputlookup hadoop_host2maxcpu

index=hadoopmon_configs source=*mapred-site.xml | kv hadoop_conf_xml | stats count last(mapred_job_tracker) as mapred_instance by host | table host mapred_instance | output lookup hadoop_host2mapred

index=hadoopmon_configs source=*core-site.xml | kv hadoop_conf_xml | stats count last(fs_default_name) as hdfs_instance by host | rex field=hdfs_instance mode=sed "s,hdfs://,," | table host hdfs_instance | output lookup hadoop_host2hdfs

The lookup behavior is unnecessarily sucky here, and the error message isn't very helpful here. We will address both issues in a future release for sure. Thanks for reaching out!

Highlighted

Re: Help on HadoopOps

Communicator

Is that because am working on CDH4?? compatibility problem??

0 Karma
Highlighted

Re: Help on HadoopOps

Splunk Employee
Splunk Employee

Likely. Hadoopops does not support yarn/MR2 yet.

0 Karma
Highlighted

Re: Help on HadoopOps

Splunk Employee
Splunk Employee

actually, most CDH4 deployments do not use YARN/MR2 yet. and many customers are using HadoopOps on CDH4. see the discussion here:

http://splunk-base.splunk.com/answers/77676/hadoop-ops-app-and-cloudera