Security

How do I improve ldapsearch performance

napomokoetle
Communicator

My environment:

Splunk Supporting Add-on for Active Directory 2.1.1
Splunk Enterprise 6.3.1
Running on Linux Centos 6.5

When I execute any LDAP search I have to wait for at least 5 minutes before I see results back! This is extremely slow. What can I do to troubleshoot this performance problem and improve the performance?

Kind regards,
J. Napo Mokoetle

the0duke0
Path Finder

It's a Festivus Miracle! sa-ldapsearch 2.2.0 was released on 1/3/2019 and it is based off of ldap3 v2.5, and it seems to have resolved the slowness issue! Go download it now!

0 Karma

datasearchninja
Communicator

I have had good results by upgrading the version of ldap3 that ships with the package. See my answer in https://answers.splunk.com/answers/587284/poor-ldapsearch-performance-sa-ldapsearch-app.html?childTo...

0 Karma

koshyk
Super Champion

any chance for the support case to be escalated? if Splunk is not interested, we can write an addon ourself

0 Karma

rtestu_splunk
Splunk Employee
Splunk Employee

Hello,

Have you tried this :

To improve performance on queries against ADs with large numbers of users, select only the query attributes you need to complete your analysis. For example, if you need just two attributes, distinguishedName and sAMAccountName, say so. Use this command:

| ldapsearch search="(objectClass=user)" attrs="distinguishedName,sAMAccountName"

instead of:

| ldapsearch search="(objectClass=user)"

It comes from the documentation :
http://docs.splunk.com/Documentation/SA-LdapSearch/latest/User/UseSA-ldapsearchtotroubleshootproblem...

I tried and it significantly improved our performance.

Hope it will help !
Romain.

the0duke0
Path Finder

This issue makes me a very sad panda.

I've had a case open with support since 12/2015 for this issue. They acknowledged that it is an issue they are working on, but still no fix. The problem started with v2 of the add-on. If you go back to the v1 version, which is Java based:-(, the performance is on par with normal LDAP queries. The downside, beyond Java, of downgrading to v1 is that there are lots of features missing. Namely Non-ASCII character support and ability to specify a BaseDN. Going back to v1 broke apps that we use and have created so we are back on slow v2. We try to use scheduled reports as much as possible to hide the slow searching, but that is not ideal, especially for developing searches.

napomokoetle
Communicator

Hi rtestu,

Thanks for the response.
I do use very specific searches. But the result is still very sluggish response. Still have to weight at last 5 minutes for any results to show up!
Here's an example query I execute:

search="(&(objectclass=user)(!(objectClass=computer))(accountExpires=9223372036854775807))"|sort sAMAccountName|table sAMAccountName,cn,userPrincipalName,userAccountControl

Could someone perhaps suggest ways to trace the situation to get to where the bottleneck could be?

0 Karma

rtestu_splunk
Splunk Employee
Splunk Employee

And did you try what I mentioned ?

| ldapsearch search="(&(objectclass=user)(!(objectClass=computer))(accountExpires=9223372036854775807))"
attrs="sAMAccountName,cn,userPrincipalName,userAccountControl"
| sort sAMAccountName
| table sAMAccountName,cn,userPrincipalName,userAccountControl

It could significantly improve the performance if you have to retrieve lots of data from the LDAP. In your case, I am not sure it will improve a lot since you seem to retrieve a few accounts ...

Romain.

0 Karma

napomokoetle
Communicator

Hi retestu,

Yes I als tried limiting the attributes as you suggested but it's not helping.

The odd thing is when I perform the search from the command line on the same search head as show below, the results return immediately.

[root@JHBTNXSPL111 ~]# ldapsearch -x -b "DC=inter,DC=Kransnetwork,DC=Net" -D "CN=SRV-TCC-Splunk LDAP,OU=Service Accounts,OU=Groups,OU=TCC,DC=inter,DC=Kransnetwork,DC=net" -w "I3m5TFfB\$uh2%ze" -h 10.10.227.150 -p 389 -A '(&(objectclass=user)(!(objectClass=computer))(accountExpires=9223372036854775807))'

Also, the Splunk solution uses LDAP integration to authenticate users and it works just fine when users use the AD credentials to log in.

So it really seems to me like the problem is with the "Splunk Supporting Add-on for Active Directory".

0 Karma

Lucas_K
Motivator

There is no doubt the root cause is the addon.

A local os based ldapsearch takes milliseconds. The addon based one takes minutes.

0 Karma

napomokoetle
Communicator

I agree 100%! There is something NOT optimal with that "Splunk Supporting Add-on for Active Directory" that makes it not perform as expected.

I've seen many other posts from folks experiencing the same problem. The app serves a very useful function, but in its current state it's unusable.

0 Karma

napomokoetle
Communicator

I suppose Splunk also has no clue on fixing this one or helping us get it right.

MuS
SplunkTrust
SplunkTrust

Hi napomokoetle,

did you open a support case?

0 Karma

napomokoetle
Communicator

Hi Mus,

I have had the problem for a very long time and don't recall opening a trouble ticket. But there are many others who have had the same problem and opened tickets but I believe they never had the problem solved.

0 Karma

gwalford
Path Finder

I am dealing with a large domain, and an LDAPsearch takes over 8 hours to complete - but it usually times out.

0 Karma

gwalford
Path Finder

I opened a ticket, it seems that there is a bug in SA-LDAPsearch that is being addressed.

napomokoetle
Communicator

Thank you gwalford! I hope this issue can be finally be resolved. Much appreciated.

0 Karma

gwalford
Path Finder

I strongly recommend you open a ticket on this issue as well.

The more customers that open tickets, the higher of a priority this becomes on the DEV side, and the more quickly you will get this issue addressed.

0 Karma

napomokoetle
Communicator

I'll do so.

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi napomokoetle,

just a quick question: do you use the LDPA Add-on or the SA-ldapsearch? I'm asking because you tagged the question with Add-on for LDAP which does not provide a ldapsearch command and listed the Add-on for Active Directory as well...

cheers, MuS

0 Karma

napomokoetle
Communicator

Hi MuS, Thanks for your response. I'm using the SA-ldapsearch. I hope you can help resolve this performance problem.
Just to check if Splunk was the problem or NOT. I installed the Centos ldapsearch client and executed the same Active Directory search I've been trying through Splunk, and the AD results return in seconds.
So it seems to me the problem is with Splunk and not Active Directly or my system.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...