Help on HadoopOps

[ncbshiva]

Hi all,
Anybody aware of these warnings in HadoopOps app?

Lookup table 'hadoop_host2maxcpu' is empty.
No matching fields exist
Lookup table 'hadoop_host2mapred' is empty.

why am getting these?
how to resolve?
Should i need to do any conf setting??

Thanks in advance.

[pierre4splunk]

These lookups are populated when the app is first initialized, and then updated once per day thereafter. Are you seeing this after a new deployment or upgrade? If so, this message could show up if some data is not yet available. The message persists until the next scheduled search updates the lookup correctly.

You can run these scheduled searches manually, both to troubleshoot and to populate the lookup directly. I've pasted them here for reference (expanded from saved searches and macros):

index=hadoopmon_os source="cpu" | stats max(CPU) as maxcpu by host | eval maxcpu = maxcpu + 1 | outputlookup hadoop_host2maxcpu

index=hadoopmon_configs source=*mapred-site.xml | kv hadoop_conf_xml | stats count last(mapred_job_tracker) as mapred_instance by host | table host mapred_instance | output lookup hadoop_host2mapred

index=hadoopmon_configs source=*core-site.xml | kv hadoop_conf_xml | stats count last(fs_default_name) as hdfs_instance by host | rex field=hdfs_instance mode=sed "s,hdfs://,," | table host hdfs_instance | output lookup hadoop_host2hdfs

The lookup behavior is unnecessarily sucky here, and the error message isn't very helpful here. We will address both issues in a future release for sure. Thanks for reaching out!

[pierre4splunk]

actually, most CDH4 deployments do not use YARN/MR2 yet. and many customers are using HadoopOps on CDH4. see the discussion here:

http://splunk-base.splunk.com/answers/77676/hadoop-ops-app-and-cloudera

[lyuanlai_splunk]

Likely. Hadoopops does not support yarn/MR2 yet.

[ncbshiva]

Is that because am working on CDH4?? compatibility problem??

[lyuanlai_splunk · Accepted solution]

This suggest that 'ps' data are not getting in from forwarder (Splunk_TA_hadoopops). Is [script://./bin/hadoopmon_ps.sh] enabled?

Edit:
As pierre4splunk replied below, core-site.xml and mapred-site.xml need to be monitored to create hdfs and mapred lookup, respectively. maxcpu lookup depends on results from hadoopmon_cpu.sh. Might be worth double-checking.

[ncbshiva]

IF ANY ONE KNOWS PLZ HELP ME