Size Limit Exceeded in splunkd.log for LDAP Authentication

[JeremyHagan]

I've just configured Splunk 6.0.3 using the GUI for AD authentication. I am able to save the configuration and map groups to roles and I can even see the group members when I do so, but no users are able to authenticate. Whenever I save the LDAP strategy name I get errors in the Splunkd.log as follows:
ScopedLDAPConnection - strategy="my_ldap" LDAP Server returned warning in search for DN="OU=Users,OU=My OU,DC=mydomain,DC=local". reason="Size limit exceeded"
ScopedLDAPConnection - strategy="my_ldap" LDAP Server returned warning in search for DN="OU=Groups,OU=My OU,DC=mydomain,DC=local". reason="Size limit exceeded"

I know for a fact that the OUs I have specified only have about 20 users and groups in them. I've tried reconfiguring the limit from 1000 up to 1,000,000 with no success. I've tried adding search filters like (objectClass=group) although this shouldn't make a difference since each OU only contains the object types I am targeting.

Is this a bug? What am I doing wrong? I have tried restarting Splunk too.

[scc00]

In my case, I was having the exact same issue. Upon review of the AD schema I found that all of the users were not sitting in the user base DN container specified. So the issue was due to the LDAP requests inability to find the users. My windows team had to go into AD and fix that. Now all my users are coming in without issue. With the configurations below.

User settings

User base DN
{UserBaseDN}

User base filter
Empty
User name attribute
{authenticationnameattribute}

Real name attribute
{displayname}

Group mapping attribute
{groupmemberattribute}

Group settings

Group base DN
{GroupBaseDN}

Static group search filter
(&(objectclass=Group)(|(sAMAccountName={targetgrouppattern1})

Group name attribute
cn

Static member attribute
member

Nested groups
false
Dynamic group settings
-empty-
Dynamic member attribute
-empty-
Dynamic group search filter
-empty-

Advanced settings

Enable referrals with anonymous bind only
Yes
Search request size limit
1000
Search request time limit
15
Network socket timeout
20

[JeremyHagan · Accepted solution]

I ended up deleting the whole scheme and starting again with reference to the example here:
http://docs.splunk.com/Documentation/Splunk/6.0.3/admin/Authenticationconf

While not materially different, I had previously named the scheme as the FQDN of the domain as the name and specified a DN as the bind user. This time I used the NETBIOS name (i.e. no dots) as the name of the scheme and use the UPN of the bind user instead of the DN.

After this it worked perfectly.

[Cbr1sg]

Hi, could you give example of how you do it to make this works? Thanks

[curtisb1024]

Have you tried turning up the log level for ScopedLDAPConnection? (Settings > System Settings > System Logging) I'd do this then reload the authentication configuration and check splunkd.log, it can yield valuable clues as to what is causing the LDAP to return so many entries.

[grijhwani]

The error being reported is from the LDAP server. By default it will only return a maximum of 1000 matching records. Your search is not targetted enough to reduce the results to 1000 entries or less. It makes no difference what you set the limit to in Splunk, it is the LDAP server truncating the return. Are you sure your User Base DN is right (originally wrote "BindDN" in error), and putting you in the right tree? Remember that User Base DN and Group Base DN and search criteria can be different. Whatever you THINK there is to return, the LDAP itself would appear to disagree (since it is that which is imposing the default 1000 entry limit). In order to find out why, you need to be investigating what your configured query actually returns, and what else there is in the tree that it returing in response to your query.

You don't specify what platform you are running on, but if it is Linux I would suggest installing ldap-utils or equivalent and tailoring your search manually with ldapsearch. Also search previous questions/answers on here. There is plenty of material already to be had.

example usage here

Our (redacted) config:

User settings

    User base DN
    {UserBaseDN}

    User base filter
    (objectclass=User)

    User name attribute
    {authenticationnameattribute}

    Real name attribute
    {displayname}

    Group mapping attribute
    {groupmemberattribute}


Group settings

    Group base DN
    {GroupBaseDN}

    Static group search filter
    (&(objectclass=Group)(|(sAMAccountName={targetgrouppattern1}*)(sAMAccountName={targetgrouppattern1}*)...etc))

    Group name attribute
    cn

    Static member attribute
    member

    Nested groups
    false

    Dynamic group settings
    -empty-

    Dynamic member attribute
    -empty-

    Dynamic group search filter
    -empty-

Advanced settings

    Enable referrals with anonymous bind only
    Yes

    Search request size limit
    1000

    Search request time limit
    15

    Network socket timeout
    20

[grijhwani]

When I mentioned BindDN I meant User base DN. And yes - if users are returned by the search, they will appear as users as long as they are members of groups mapped to at least one role.

Have you consulted logs on the LDAP side to see what it thinks is being returned. I'd be inclined to attack this from the LDAP side initially, seeing as that is what is pushing a glut of entries and truncating it.

In our LDAP tree DOES contain thousands of users, but a search filter limits the scope to only those groups we are explicitly interested in.

[grijhwani]

Intriguing problem you have. I will take a look at our LDAP config tomorrow to see what suggests itself.

[JeremyHagan]

Hi,

As I said, the Group and User search bases are both restricted to OUs that only have those object type in them and they only have about 20 or so objects in each OU. There simply aren't 1,000 objects to return.

The Bind DN is in the same AD domain, but not in either of the OUs specified for the Group and User search base. And in any case the LDAP bind is obviously working because I can map groups to roles. I just can't authenticate the users that are in the groups.

Splunk is running on Windows Server 2008 R2.

Should the users that are enumerated show up under "Users" in Splunk?