All Apps and Add-ons

How to speed up LDAP / Active Directory searches, specifically Asset or Identity lookups?

Richfez
SplunkTrust
SplunkTrust

My AD query for Asset or Identity list generation takes a long time to run. Is there any way to speed it up?

For instance, suppose my version of the search Splunk supplies as a starting point looks like this:

| ldapsearch search="(&(objectClass=user) (!(objectClass=computer)))" 
| search userAccountControl="NORMAL_ACCOUNT"
| eval suffix=""
| eval priority=case(       like(distinguishedName,"%OU=Expired,DC=MyDC,DC=local"), "critical",
                            like(distinguishedName,"%OU=Vendors,DC=MyDC,DC=local"), "critical",
                            like(distinguishedName,"%OU=Support,DC=MyDC,DC=local"), "high",
                            like(distinguishedName,"%OU=SYSTEMS,DC=MyDC,DC=local"), "medium",
                            like(distinguishedName,"%OU=Users,DC=MyDC,DC=local"), "low",
                            1==1,"medium" )
| eval category=case(       like(distinguishedName,"%OU=Expired,DC=MyDC,DC=local"), "expired",
                            like(distinguishedName,"%OU=Vendors,DC=MyDC,DC=local"), "vendors",
                            1==1,"normal" )
| eval watchlist="false"
| eval endDate=if(accountExpires="(never)","",accountExpires)
| rex field=manager "CN=(?<managedBy>[^,]*)"
| table sAMAccountName, personalTitle, displayName, givenName, sn, suffix, mail, telephoneNumber, mobile, 
  managedBy, priority, department, category, watchlist, whenCreated, endDate 
| rename sAMAccountName as identity, personalTitle as prefix, displayName as nick, givenName as first, 
  sn as last, mail as email, telephoneNumber as phone, mobile as phone2, department as bunit, whenCreated as startDate
| outputlookup MyCompany_identities

My current search takes 400 seconds (+/- 5 seconds) to finish. How can I make this faster?

1 Solution

Richfez
SplunkTrust
SplunkTrust

For others who may need to do this as well, I've found an order of magnitude speed up in LDAP queries to AD if you limit your return fields to the fields you actually need.

Without that the query returns every field in AD, a very large and lengthy return set. By using attrs="attribute1,attribute2..." to only return the list you need, you can make an order of magnitude difference in speed.

| ldapsearch search="(&(objectClass=user) (!(objectClass=computer)))" attrs="userAccountControl,sAMAccountName,personalTitle,displayName,givenName,sn,mail,telephoneNumber,mobile,manager,department,whenCreated,accountExpires"
| search userAccountControl="NORMAL_ACCOUNT"
| eval suffix=""
...

Before, original search ran in 400 seconds. The new search returns in less than 40 (averages about 36). That is a very significant increase.

For what it's worth, my hand was forced one day when I was debugging something with this output but getting SO aggravated with how long the searches took. I was looking through all the fields returned and realized that "Wait, there's a way to only return what you need. I wonder if that will speed this up somewhat?" I was astounded when I got that big of an improvement.

View solution in original post

n0str0m08
Explorer

I know that this question was answered a long time ago. However, this additional solution can be helpful for other Splunkers to speed the search up.

According to the second example on Microsoft Wiki which is explained in Note 1. The filter (sAMAccountType=805306368) can be used to search all user objects in AD. It is equal to (&(objectCategory=person)(objectClass=user)) and more efficient, however harder to remember.

0 Karma

Richfez
SplunkTrust
SplunkTrust

For others who may need to do this as well, I've found an order of magnitude speed up in LDAP queries to AD if you limit your return fields to the fields you actually need.

Without that the query returns every field in AD, a very large and lengthy return set. By using attrs="attribute1,attribute2..." to only return the list you need, you can make an order of magnitude difference in speed.

| ldapsearch search="(&(objectClass=user) (!(objectClass=computer)))" attrs="userAccountControl,sAMAccountName,personalTitle,displayName,givenName,sn,mail,telephoneNumber,mobile,manager,department,whenCreated,accountExpires"
| search userAccountControl="NORMAL_ACCOUNT"
| eval suffix=""
...

Before, original search ran in 400 seconds. The new search returns in less than 40 (averages about 36). That is a very significant increase.

For what it's worth, my hand was forced one day when I was debugging something with this output but getting SO aggravated with how long the searches took. I was looking through all the fields returned and realized that "Wait, there's a way to only return what you need. I wonder if that will speed this up somewhat?" I was astounded when I got that big of an improvement.

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...