All Apps and Add-ons

How to speed up LDAP / Active Directory searches, specifically Asset or Identity lookups?

SplunkTrust
SplunkTrust

My AD query for Asset or Identity list generation takes a long time to run. Is there any way to speed it up?

For instance, suppose my version of the search Splunk supplies as a starting point looks like this:

| ldapsearch search="(&(objectClass=user) (!(objectClass=computer)))" 
| search userAccountControl="NORMAL_ACCOUNT"
| eval suffix=""
| eval priority=case(       like(distinguishedName,"%OU=Expired,DC=MyDC,DC=local"), "critical",
                            like(distinguishedName,"%OU=Vendors,DC=MyDC,DC=local"), "critical",
                            like(distinguishedName,"%OU=Support,DC=MyDC,DC=local"), "high",
                            like(distinguishedName,"%OU=SYSTEMS,DC=MyDC,DC=local"), "medium",
                            like(distinguishedName,"%OU=Users,DC=MyDC,DC=local"), "low",
                            1==1,"medium" )
| eval category=case(       like(distinguishedName,"%OU=Expired,DC=MyDC,DC=local"), "expired",
                            like(distinguishedName,"%OU=Vendors,DC=MyDC,DC=local"), "vendors",
                            1==1,"normal" )
| eval watchlist="false"
| eval endDate=if(accountExpires="(never)","",accountExpires)
| rex field=manager "CN=(?<managedBy>[^,]*)"
| table sAMAccountName, personalTitle, displayName, givenName, sn, suffix, mail, telephoneNumber, mobile, 
  managedBy, priority, department, category, watchlist, whenCreated, endDate 
| rename sAMAccountName as identity, personalTitle as prefix, displayName as nick, givenName as first, 
  sn as last, mail as email, telephoneNumber as phone, mobile as phone2, department as bunit, whenCreated as startDate
| outputlookup MyCompany_identities

My current search takes 400 seconds (+/- 5 seconds) to finish. How can I make this faster?

1 Solution

SplunkTrust
SplunkTrust

For others who may need to do this as well, I've found an order of magnitude speed up in LDAP queries to AD if you limit your return fields to the fields you actually need.

Without that the query returns every field in AD, a very large and lengthy return set. By using attrs="attribute1,attribute2..." to only return the list you need, you can make an order of magnitude difference in speed.

| ldapsearch search="(&(objectClass=user) (!(objectClass=computer)))" attrs="userAccountControl,sAMAccountName,personalTitle,displayName,givenName,sn,mail,telephoneNumber,mobile,manager,department,whenCreated,accountExpires"
| search userAccountControl="NORMAL_ACCOUNT"
| eval suffix=""
...

Before, original search ran in 400 seconds. The new search returns in less than 40 (averages about 36). That is a very significant increase.

For what it's worth, my hand was forced one day when I was debugging something with this output but getting SO aggravated with how long the searches took. I was looking through all the fields returned and realized that "Wait, there's a way to only return what you need. I wonder if that will speed this up somewhat?" I was astounded when I got that big of an improvement.

View solution in original post

Explorer

I know that this question was answered a long time ago. However, this additional solution can be helpful for other Splunkers to speed the search up.

According to the second example on Microsoft Wiki which is explained in Note 1. The filter (sAMAccountType=805306368) can be used to search all user objects in AD. It is equal to (&(objectCategory=person)(objectClass=user)) and more efficient, however harder to remember.

0 Karma

SplunkTrust
SplunkTrust

For others who may need to do this as well, I've found an order of magnitude speed up in LDAP queries to AD if you limit your return fields to the fields you actually need.

Without that the query returns every field in AD, a very large and lengthy return set. By using attrs="attribute1,attribute2..." to only return the list you need, you can make an order of magnitude difference in speed.

| ldapsearch search="(&(objectClass=user) (!(objectClass=computer)))" attrs="userAccountControl,sAMAccountName,personalTitle,displayName,givenName,sn,mail,telephoneNumber,mobile,manager,department,whenCreated,accountExpires"
| search userAccountControl="NORMAL_ACCOUNT"
| eval suffix=""
...

Before, original search ran in 400 seconds. The new search returns in less than 40 (averages about 36). That is a very significant increase.

For what it's worth, my hand was forced one day when I was debugging something with this output but getting SO aggravated with how long the searches took. I was looking through all the fields returned and realized that "Wait, there's a way to only return what you need. I wonder if that will speed this up somewhat?" I was astounded when I got that big of an improvement.

View solution in original post