topic Re: Help on HadoopOps in All Apps and Add-ons

Help on HadoopOps

ncbshiva — Mon, 28 Sep 2020 14:02:31 GMT

Hi all,
Anybody aware of these warnings in HadoopOps app?

Lookup table 'hadoop_host2maxcpu' is empty.
No matching fields exist
Lookup table 'hadoop_host2mapred' is empty.

why am getting these?
how to resolve?
Should i need to do any conf setting??

Thanks in advance.

Re: Help on HadoopOps

ncbshiva — Thu, 06 Jun 2013 08:47:22 GMT

IF ANY ONE KNOWS PLZ HELP ME

Re: Help on HadoopOps

lyuanlai_splunk — Thu, 06 Jun 2013 20:55:32 GMT

This suggest that 'ps' data are not getting in from forwarder (Splunk_TA_hadoopops). Is [script://./bin/hadoopmon_ps.sh] enabled?

Edit:
As pierre4splunk replied below, core-site.xml and mapred-site.xml need to be monitored to create hdfs and mapred lookup, respectively. maxcpu lookup depends on results from hadoopmon_cpu.sh. Might be worth double-checking.

Re: Help on HadoopOps

pierre4splunk — Fri, 07 Jun 2013 03:24:06 GMT

These lookups are populated when the app is first initialized, and then updated once per day thereafter. Are you seeing this after a new deployment or upgrade? If so, this message could show up if some data is not yet available. The message persists until the next scheduled search updates the lookup correctly.

You can run these scheduled searches manually, both to troubleshoot and to populate the lookup directly. I've pasted them here for reference (expanded from saved searches and macros):

index=hadoopmon_os source="cpu" | stats max(CPU) as maxcpu by host | eval maxcpu = maxcpu + 1 | outputlookup hadoop_host2maxcpu

index=hadoopmon_configs source=*mapred-site.xml | kv hadoop_conf_xml | stats count last(mapred_job_tracker) as mapred_instance by host | table host mapred_instance | output lookup hadoop_host2mapred

index=hadoopmon_configs source=*core-site.xml | kv hadoop_conf_xml | stats count last(fs_default_name) as hdfs_instance by host | rex field=hdfs_instance mode=sed "s,hdfs://,," | table host hdfs_instance | output lookup hadoop_host2hdfs

The lookup behavior is unnecessarily sucky here, and the error message isn't very helpful here. We will address both issues in a future release for sure. Thanks for reaching out!

Re: Help on HadoopOps

ncbshiva — Fri, 07 Jun 2013 10:50:04 GMT

Is that because am working on CDH4?? compatibility problem??

Re: Help on HadoopOps

lyuanlai_splunk — Tue, 11 Jun 2013 04:21:01 GMT

Likely. Hadoopops does not support yarn/MR2 yet.

Re: Help on HadoopOps

pierre4splunk — Tue, 11 Jun 2013 20:50:50 GMT

actually, most CDH4 deployments do not use YARN/MR2 yet. and many customers are using HadoopOps on CDH4. see the discussion here:

http://splunk-base.splunk.com/answers/77676/hadoop-ops-app-and-cloudera