[LDAP] Private user dashboards (views) are suddenly not visible to admin user.

[sylim_splunk]

I was able to manage/view XML for private user dashboards till last friday. Since today I realised that I am not able to even list those private dashboards under Settings > UI > Views, OR, Settings > All Configs.

As platform admins, we promote user made objects to global sharing.

I compared with my teammates listed below, we all share the same role (admin) and yet they are able to view those same private dashboards that I cannot view/list.

SH Version 8.0.5 in SHC.

[sylim_splunk · Accepted solution]

It turned out to be caused by the combination of LB source IP stickiness and LDAP user cache. LDAP user cache is not replicated across the SHCluster members and having different cache data (user data).

The user that created a private dashboards were not cached in the search head that the admin is logged on due to the insufficient cache size while other admins might have logged on to the search head where the user was created in its cache.

The LDAP user cache is 1000 by default - this may sound enough considering the active users but it's not about the number of entries for user but the number of entries for group - one user with 100s groups can occupy as many slots in the cache.

- To increase the cache size find the config parameter below;
in limits.conf
[ldap]
max_users_to_precache = 100000

To prove it hits the same issue, ask the user  to log-in to the search head you are logged in to - this will trigger the user to be populated in the cache of the SH and see if you can find the user's dashboard.