LDAP Size Limit

[maciep]

Hi all,

I know this has been covered before here, but I'm still struggling with it. How do we tell Splunk to return more than 1000 users per LDAP strategy. I've tried increasing the size limit and the max number of precache users, but that hasn't helped.

For example, I have a strategy right now that's filtering for 4 groups with a total number of users (nested) at about 1200. My size limit for the strategy is 5,000 and precahce users is 10,000. But I'm still receiving the Size Limit warning in my logs for that strategy.

And I know the size limit configuration mentions "The number actually returned is subject to the limit imposed by the LDAP server." But that just doesn't make sense to me. Every other technology that queries AD can get around those limits. I feel like Splunk should be able to as well.

Is there something I'm missing or just not understanding? It's frustrating because we just have to keep creating more and more strategies. If that's the only way, then that's what we'll do (assuming every group has less than 1000 members). But I hope there's another way now or will be in the future.

We are currently on Splunk 6.0

Thanks for any insight!

[raziasaduddin]

Splunk support basically told me to increase the limit on our AD servers, which is not an option for a large enterprise. Besides, that just introduces extra load on the AD servers for all the other queries.

We are still having the same problem. I tried manually editing the authentication.conf to add the group mapping for the 8000 user group, but users are still unable to login. Debugging the LDAP logging did not help.

That pageSize option needs to be reintroduced back into Splunk. Most other apps have that to accommodate for this.

[raziasaduddin]

I got my support contact to agree to submit an ER for this.

[maciep]

Exactly, we don't really have that option to increase the limit in AD. I wish Splunk worked like most other apps I've dealt with in the past (still need to open enhancement request). In a nutshell:

Just check AD when a user tries to log in. If the user should have access, check to see if their account exists. If they should but not created in Splunk yet, create it then. If it does exist, verify roles/etc based on current membership and update splunk account accordingly.

There's no need query AD for every user at once...especially if you're unwilling to modify the page size.

[antlefebvre]

I assume you are using Active Directory. You will want to change MaxPageSize. Default is 1000.

http://support.microsoft.com/kb/315071

[maciep]

antlefebvre, it may be Splunk's only solution. But my scripts (powershell, vbscript, kix, etc) can all bring back more than 1000 objects per search. So speaking of Active Directory in general, it's not the only solution.

[my_splunk]

It is absurd that Splunk has not ldap pagining....To modify AD in a complex architecture can make damage on performance and other application; also i do not have administration of AD ....

[antlefebvre]

It isn't a solution, it is THE solution. You will run into that issue if you use an LDAP browser or any other tool that requires LDAP integration. You need to fix it from the LDAP server, not the LDAP client.

Try downloading Softerra LDAP Browser. Query your OUs/groups that have over 1000 members and you will see the issue.

[maciep]

I understand that is a solution, but it's not going to happen in our environment. And if that is Splunk's only solution, then I need to open an ehancement request, because they need to improve the way they do ldap integration.

[my_splunk]

I'm in a problem like this from days, so i asked to windows administrators to change maximun page size on Active Directory and now i'm waiting for this. However it is a vary strange regression of Splunk ...In 3.x version it was present a pageSize directive for ldap pagining search.