I've noticed for a while that the SA-LDAPsearch "ldapfilter" commands can be incredibly slow in obtaining results. The process may be at 100% CPU, not generating any errors, but, just takes several minutes to come back with results, even for pretty small searches.
index=logins Success| ldapfilter domain=mydomain search="(cn=$user$)" attrs="company,title,division,displayName,department,mail"
Running this with a timeline of "today" takes so long that if I don't leave my browser window open on this tab, the job times out - but running the command without the ldapfilter returns 19 hits in only a second or two.
And checking the ps output shows:
splunk 14436 94.9 0.0 133420 57488 ? Rs 14:29 0:12 /opt/splunk/bin/python /opt/splunk/etc/apps/SA-ldapsearch/bin/ldapfilter.py __EXECUTE__ domain=mydomain search="(cn=$user$)" attrs="company,title,division,displayName,department,mail"
Running tcpdump on the Search Head shows the LDAP connections appearing in bursts. Results eventually appear, and everything is correct in them, but, the actual time that it takes to get results is massive.
From the Job Inspector, first column being seconds:
1,572.10 command.ldapfilter 57 19 19
This seems a bit insane. Especially for something which is only a handful of invocations and results - is there something I could be tuning to make this more tolerable?
This is with the latest SA-Ldapsearch as well (2.1.4).
Hi All, Just an update, on Jan. 3rd, Splunk released v2.20 of SA-ldapsearch. This has the fix for the performance issues by integration of newer libraries. You can see the notes in "What's New" of the Release Notes:
Hopefully this should resolve any issues with performance people were facing, already heard good feedback from others.
I have had good results by upgrading the version of ldap3 that ships with the package. See my answer in https://answers.splunk.com/answers/587284/poor-ldapsearch-performance-sa-ldapsearch-app.html?childTo...
It is not the SA-ldap which is slow, but Microsoft AD itself is slow unless we put in maximum conditions and NOT conditions too.
Have a try with below type of Query (put in domain, object class, NOT object class etc
|ldapsearch domain=ADMIN search="(&(objectClass=user)(!(objectClass=computer)(cn=$user$)))"
2nd option is to do ldapsearch on whole of the userlist/groups , once in a day and store is as a lookup file. This is what we do and is quite fast after the 1st time.
I don't think AD is slow. We experience very long reply times too, but only since we updated to SA-ldapsearch v2.4.1. On one test machine, we still have the old version (v1.1.10) running, and the same query to the same AD servers is constantly much faster, depending on the query 2-10 times faster.
This is absolutely an issue with SA-ldapsearch and has been present since V2; but with java issues and security holes I can't roll back to V1 which works fine. I really wish splunk would fix this.
Splunk support confirmed that v2 is slower and it's in the release notes too ( https://docs.splunk.com/Documentation/SA-LdapSearch/2.1.4/User/ReleaseNotes -> TAG-9567 - but didn't expect it to be so much slower). According to Splunk Support, they have no intention to fix this :-[ But they continue to support v1 for those folks who can live with java. We are considering to go back to v1.