The DMC general setup does not work if you delete or rename the admin account (e.g. via user-seed.conf).
http://docs.splunk.com/Documentation/Splunk/latest/Admin/User-seedconf
In 6.2, the work-around is to change the owner = nobody for all knowledge objects within the metadata/local.meta file of the splunk_management_console app, and then executing a splunk restart or debug/refresh.
In 6.3, this does not work.
What is the work-around/fix for this issue?
Create a new "admin" user account and assign it to a new role that has no privileges.
Create a new "admin" user account and assign it to a new role that has no privileges.
This issue has been identified as a product defect - internal reference: SPL-92633.
The problem is quite simply that some DMC actions (typically, configuration changes) are hard-coded to run lookup-manipulating searches as the "admin" user, which of course fails if the user in question has been renamed.
The work-around (and actually, the fix too) is to leverage the dispatchAs = user
property in savedsearches.conf (new to 6.2) which allows a saved search to be run as the invoking user instead of the owning user when called.
Work-around steps:
dispatchAs = user
key to the DMC Asset - Build Full
saved search stanza in $SPLUNK_HOME/etc/apps/splunk_management_console/local/savedsearches.conf
Thanks hexx. Unfortunately, this workaround/fix did not work for me.
I made the changes per your steps (and removed my local.meta changes), but I continue to get ldap calls for the admin user, and the modal screen does not appear. I also added dispatchAs = user to all of the savedsearches stanzas that are in default, but same thing happened. I even went so far as to add dispatchAs = user to a default stanza in this savedsearches.conf, but still no luck. Also, changing the owner in local.meta to a renamed admin account does not work. Lastly, I removed LDAP authentication, and that did not help.
In addition, the Forwarder Monitoring Setup page does not load when the "admin" user account does not exist.
So far, the only thing that has worked for me is to temporarily add a local "admin" user account.
Is there a log.cfg setting that I can set to DEBUG the calls to which populating lookup search is run, and by what user?
I'm sorry to hear this suggested work-around did not function. I would like to strongly encourage you to open a support case so that we can look into this issue in more detail and identify if there is a new defect to be fixed here.
There was a specific issue with the DMC setup and renamed admin accounts that was fixed in 6.3. Can you describe in detail what interactions with the DMC are no longer working and how that manifests itself?
When changing to a Distributed configuration and clicking Apply Changes (with no errors), the Modal screen fails to appear or apply any changes. Only after creating the 'admin' account, the changes apply as expected. Also, splunkd.log shows failed admin ldap logins.
Actually, I was wrong: The fix for this issue did not make it into 6.3 which explains why you are still seeing it! I will explain how to work around this problem in an answer.