How do I improve ldapsearch performance

[napomokoetle]

My environment:

Splunk Supporting Add-on for Active Directory 2.1.1
Splunk Enterprise 6.3.1
Running on Linux Centos 6.5

When I execute any LDAP search I have to wait for at least 5 minutes before I see results back! This is extremely slow. What can I do to troubleshoot this performance problem and improve the performance?

Kind regards,
J. Napo Mokoetle

[the0duke0]

It's a Festivus Miracle! sa-ldapsearch 2.2.0 was released on 1/3/2019 and it is based off of ldap3 v2.5, and it seems to have resolved the slowness issue! Go download it now!

[datasearchninja]

I have had good results by upgrading the version of ldap3 that ships with the package. See my answer in https://answers.splunk.com/answers/587284/poor-ldapsearch-performance-sa-ldapsearch-app.html?childTo...

[koshyk]

any chance for the support case to be escalated? if Splunk is not interested, we can write an addon ourself

[rtestu_splunk]

Hello,

Have you tried this :

To improve performance on queries against ADs with large numbers of users, select only the query attributes you need to complete your analysis. For example, if you need just two attributes, distinguishedName and sAMAccountName, say so. Use this command:

| ldapsearch search="(objectClass=user)" attrs="distinguishedName,sAMAccountName"

instead of:

| ldapsearch search="(objectClass=user)"

It comes from the documentation :
http://docs.splunk.com/Documentation/SA-LdapSearch/latest/User/UseSA-ldapsearchtotroubleshootproblem...

I tried and it significantly improved our performance.

Hope it will help !
Romain.

[the0duke0]

This issue makes me a very sad panda.

I've had a case open with support since 12/2015 for this issue. They acknowledged that it is an issue they are working on, but still no fix. The problem started with v2 of the add-on. If you go back to the v1 version, which is Java based:-(, the performance is on par with normal LDAP queries. The downside, beyond Java, of downgrading to v1 is that there are lots of features missing. Namely Non-ASCII character support and ability to specify a BaseDN. Going back to v1 broke apps that we use and have created so we are back on slow v2. We try to use scheduled reports as much as possible to hide the slow searching, but that is not ideal, especially for developing searches.

[napomokoetle]

Hi rtestu,

Thanks for the response.
I do use very specific searches. But the result is still very sluggish response. Still have to weight at last 5 minutes for any results to show up!
Here's an example query I execute:

search="(&(objectclass=user)(!(objectClass=computer))(accountExpires=9223372036854775807))"|sort sAMAccountName|table sAMAccountName,cn,userPrincipalName,userAccountControl

Could someone perhaps suggest ways to trace the situation to get to where the bottleneck could be?

[rtestu_splunk]

And did you try what I mentioned ?

| ldapsearch search="(&(objectclass=user)(!(objectClass=computer))(accountExpires=9223372036854775807))"
attrs="sAMAccountName,cn,userPrincipalName,userAccountControl"
| sort sAMAccountName
| table sAMAccountName,cn,userPrincipalName,userAccountControl

It could significantly improve the performance if you have to retrieve lots of data from the LDAP. In your case, I am not sure it will improve a lot since you seem to retrieve a few accounts ...

Romain.

[napomokoetle]

Hi retestu,

Yes I als tried limiting the attributes as you suggested but it's not helping.

The odd thing is when I perform the search from the command line on the same search head as show below, the results return immediately.

[root@JHBTNXSPL111 ~]# ldapsearch -x -b "DC=inter,DC=Kransnetwork,DC=Net" -D "CN=SRV-TCC-Splunk LDAP,OU=Service Accounts,OU=Groups,OU=TCC,DC=inter,DC=Kransnetwork,DC=net" -w "I3m5TFfB\$uh2%ze" -h 10.10.227.150 -p 389 -A '(&(objectclass=user)(!(objectClass=computer))(accountExpires=9223372036854775807))'

Also, the Splunk solution uses LDAP integration to authenticate users and it works just fine when users use the AD credentials to log in.

So it really seems to me like the problem is with the "Splunk Supporting Add-on for Active Directory".

[Lucas_K]

There is no doubt the root cause is the addon.

A local os based ldapsearch takes milliseconds. The addon based one takes minutes.

[napomokoetle]

I agree 100%! There is something NOT optimal with that "Splunk Supporting Add-on for Active Directory" that makes it not perform as expected.

I've seen many other posts from folks experiencing the same problem. The app serves a very useful function, but in its current state it's unusable.

[napomokoetle]

I suppose Splunk also has no clue on fixing this one or helping us get it right.

[MuS]

Hi napomokoetle,

did you open a support case?

[napomokoetle]

Hi Mus,

I have had the problem for a very long time and don't recall opening a trouble ticket. But there are many others who have had the same problem and opened tickets but I believe they never had the problem solved.

[gwalford]

I am dealing with a large domain, and an LDAPsearch takes over 8 hours to complete - but it usually times out.

[gwalford]

I opened a ticket, it seems that there is a bug in SA-LDAPsearch that is being addressed.

[napomokoetle]

Thank you gwalford! I hope this issue can be finally be resolved. Much appreciated.

[gwalford]

I strongly recommend you open a ticket on this issue as well.

The more customers that open tickets, the higher of a priority this becomes on the DEV side, and the more quickly you will get this issue addressed.

[napomokoetle]

I'll do so.

[MuS]

Hi napomokoetle,

just a quick question: do you use the LDPA Add-on or the SA-ldapsearch? I'm asking because you tagged the question with Add-on for LDAP which does not provide a ldapsearch command and listed the Add-on for Active Directory as well...

cheers, MuS

[napomokoetle]

Hi MuS, Thanks for your response. I'm using the SA-ldapsearch. I hope you can help resolve this performance problem.
Just to check if Splunk was the problem or NOT. I installed the Centos ldapsearch client and executed the same Active Directory search I've been trying through Splunk, and the AD results return in seconds.
So it seems to me the problem is with Splunk and not Active Directly or my system.