All Apps and Add-ons

Failed to start MapReduce job. Please consult search.log

Ledion_Bitincka
Splunk Employee
Splunk Employee

Running into an issue with Hunk against our Kerberized, YARN cluster. The Hunk principal auth works fine, what ends up happening is that a search will kick off, start searching thru events returns some results and previews: but after running over ~1 minute, it dies with:

01-27-2014 18:56:39.021 ERROR ERP.redacted-provider -  UserGroupInformation - PriviledgedActionException 
as:hunk@REDACTED.COM (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - UNKNOWN_SERVER)]

the following error message is thrown in the UI too

[redacted-provider] JobStartException - Failed to start MapReduce job. Please consult search.log for more information. Message: [ Failed to start MapReduce job, name=SPLK_search1.dev.hunk.redacted.com_1390848963.3_0 ] and [ null ]

Any pointers would be appreciated

Tags (3)
0 Karma
1 Solution

Ledion_Bitincka
Splunk Employee
Splunk Employee

The root cause of the above problem was a typo in the service principal specification for the resourcemananager principal

vix.yarn.resourcemanager.principal = yarn/namenode.redacted.com@REDACTED.COM

a change to the generic/wildcard principal specification fixed the issue

vix.yarn.resourcemanager.principal = yarn/_HOST@REDACTED.COM

View solution in original post

0 Karma

Ledion_Bitincka
Splunk Employee
Splunk Employee

The root cause of the above problem was a typo in the service principal specification for the resourcemananager principal

vix.yarn.resourcemanager.principal = yarn/namenode.redacted.com@REDACTED.COM

a change to the generic/wildcard principal specification fixed the issue

vix.yarn.resourcemanager.principal = yarn/_HOST@REDACTED.COM
0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...