Why is a web app using the Splunk Javascript SDK unable to retrieve a list of jobs from a member of a search head cluster?

[kmugglet]

A 3rd party developer has created a web app using the Splunk Javascript SDK to query Splunk search heads.
When app connects to a search head which is part of a Search Head Cluster (SHC), it cannot retrieve a list of jobs. Therefore the app fails.

I removed 1 SH from the cluster and disabled it from SHC as per the documentation.
The javascript then works correctly when accessing that standalone SH via the REST API and Jacascript SDK.
When pointed to the REST port of a remaining member of the SHC, the script once again fails to return a jobs list.

This is the error returned when accessing the jobs list.

TypeError: Cannot read property 'owner' of undefined 
at Object.root.namespaceFromProperties (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/utils.js:396:29) 
at root.Jobs.root.Collection.extend.instantiateEntity (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/service.js:3558:41) 
at null.instantiateEntity (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/utils.js:47:23) 
at root.Collection.root.Resource.extend._load (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/service.js:1257:35) 
at prototype.(anonymous function) (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/jquery.class.js:39:28) 
at null.<anonymous> (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/utils.js:47:23) 
at null._load (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/utils.js:47:23) 
at /data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/service.js:1342:26 
at reloginIfNecessary (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/context.js:199:30) 
at wrappedCallback (/data/snpm/SNPM-hg/savvi/web/savvi_nodejs/node_modules/splunk-sdk/lib/http.js:176:21) 

However, the app is able to create search jobs which complete successfully on both standalone and clustered SHs.
But on the clustered SHs it cannot tell when the job is completed.

I have tried multiple variations on app owner etc etc.

Basic jobs test is is below

var splunkjs = require('splunk-sdk');

// Create a Service instance and log in
var service = new splunkjs.Service({
    username:"valid_username",
    password:"valid_password",
    scheme:"https",
    host:"searchhead.cluster.xyz.com",
    port:"8089"
});

// Print installed apps to the console to verify login
service.jobs().fetch(function(err, jobs) {
    if (err) {
      console.log("Error retrieving jobs: ", err);
    return;
}

console.log("Jobs:");

var jobsList = jobs.list();
for(var i = 0; i < jobsList.length; i++) {
    var job = jobsList[i];
    console.log("  Job " + i + ": " + job.name);
    }
});

Anyone seen this before???

[kmugglet · Accepted solution]

We raised this as a case with Splunk Support and it was fixed under 6.2.5 release

SPL-103012  Undefined props.acl causes Javascript SDK to error and stop executing script.

[kmugglet]

I can query the clustered SHs using a normal rest call via curl, and get a full list of jobs as expected.

curl -k -u valid_username:valid_password https://search.cluster.xyz.com:8089/servicesNS/admin/apm_snpm/search/jobs/

[kmugglet]

More fun,
I changed the jobs list section of the code to this :

var jobs = service.jobs({owner: "app_owner", app: "app_name"});
jobs.fetch({count:200},function(err, searches) {
    console.log(jobs.list());

If I specify the number of jobs to return using the count it works, as long as the count is lower than the actual number of jobs (hard to nail down as jobs are running and expiring, but I think it's N-1, where N is the current number of jobs available to job management)