Security

How does Splunk manage LDAP or AD user-created objects if the user is no longer active?

Splunk Employee
Splunk Employee

I've got some users who are no longer around in my Splunk instance and I want to remove the user created objects. Is there a procedure I can follow for this task?

1 Solution

Splunk Employee
Splunk Employee

The intersection of LDAP and Splunk users is a challenge when it comes to user-created objects, as Splunk won't clean-out user folders or other objects if their authentication fails.

-- splunkd.log errors seen--
ERROR UserManagerPro - Failed to get LDAP user="myuser" from any configured servers
ERROR AuthenticationManagerLDAP - Could not find user="my
user" with strategy="LDAPorAD_config"
ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/search/splunk/etc/users/$userid/user-prefs/metadata/local.meta: No such file or directory
-- end splunkd.log example --

  1. Back up the splunk/etc/users/$userid folder.
  2. Search the files under splunk/etc/apps/ for the $userid string, which should show if the userid in question has ownership of a search or object that was elevated to Global permissions.
  3. Note the files and paths (look for splunk/etc/apps/*/metadata/local.meta) and search inside them for the userid. Change the owner to an admin user or yourself or to a maintenance account. Whichever you find easier as long as the account can authenticate properly.
  4. Restart Splunk services on the host.
  5. Check splunkd.log on the search head to validate the LDAP auth errors have subsided.
  6. Remove the splunk/etc/users/$userid folder.

View solution in original post

Splunk Employee
Splunk Employee

Please note that with the release of Splunk 6.6, there's a feature to find, alert, and manage orphaned knowledge objects. The details are available in the documentation here. As an admin, you'd receive a daily alert, have a dashboard to review the objects, and an opportunity to reassign them to a different user. Reviewing the steps above, the "Orphaned Scheduled Searches, Reports, and Alerts" dashboard and UI to manage objects would replace Steps 2, 3, and 4 for most use-cases. Enjoy!

Splunk Employee
Splunk Employee

If you see many errors about missing user in the splunkd.log, this is because deleted LDAP users still own objects in splunk, by example a scheduled search.
and you should clean it
Delete the objects/profile or migrate them to another user or an app. See answer below.

0 Karma

Splunk Employee
Splunk Employee

The intersection of LDAP and Splunk users is a challenge when it comes to user-created objects, as Splunk won't clean-out user folders or other objects if their authentication fails.

-- splunkd.log errors seen--
ERROR UserManagerPro - Failed to get LDAP user="myuser" from any configured servers
ERROR AuthenticationManagerLDAP - Could not find user="my
user" with strategy="LDAPorAD_config"
ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/search/splunk/etc/users/$userid/user-prefs/metadata/local.meta: No such file or directory
-- end splunkd.log example --

  1. Back up the splunk/etc/users/$userid folder.
  2. Search the files under splunk/etc/apps/ for the $userid string, which should show if the userid in question has ownership of a search or object that was elevated to Global permissions.
  3. Note the files and paths (look for splunk/etc/apps/*/metadata/local.meta) and search inside them for the userid. Change the owner to an admin user or yourself or to a maintenance account. Whichever you find easier as long as the account can authenticate properly.
  4. Restart Splunk services on the host.
  5. Check splunkd.log on the search head to validate the LDAP auth errors have subsided.
  6. Remove the splunk/etc/users/$userid folder.

View solution in original post

Explorer

What about Splunk cloud users? Is this something that cloud ops needs to handle or can these items be migrated through the UI?

Communicator

Seeing as you don't have access to the configuration files as part of the Splunk Cloud SaaS solution - then it is 100% something Splunk should take care of. They SHOULD have alerting in place that notifies them when the error message comes up in the log entries, then resolve it - but I wouldn't be surprised if you also have to raise an incident to formally ask them to remove the user dir.

0 Karma

Splunk Employee
Splunk Employee

For those of you on *Nix machines would be able to do something like this:

cd $SPLUNK_HOME/etc

for x in `find . -name 'local.meta'`; do cp $x $x.old ; sed s/olduser/newuser/ < $x > $x.new ; mv -f $x.new $x ; done

This will make a backup of the local.meta, swap out the olduser for the newuser and copy it over the local.meta. All you should have to do is restart Splunk.

Brian

Communicator

I wrapped this up in a script and it works perfectly - really nice solution to the problem here, bit surprised Splunk haven't got something out of the box for you to fix this problem. We had a set of administrators leave and this caused some pain.

The only addition I would make. is automating the removal of the user id folder as explained above (after creating a backup) then perform a refresh (as mentioned in https://answers.splunk.com/answers/168898/how-can-we-find-all-the-searches-alerts-dashboard.html) for the config settings to take affect - I didn't need to restart the splunk services for this to work.

Warning: Make sure you test this in development environments first, has the chance to be very costly when overwriting so many config files in bulk, even if temporarily until it can be reverted.

Communicator

This is money

0 Karma