How to populate Assets and Identities in ES with SA-LDAPSearch on Heavy Forwarder?

[b_chris21]

Hello,

I have a Splunk ES instance on AWS. All logs are forwarded there from a Splunk HF (full forwarding - no indexing) which collects Active Directory data. Domain is accessible only via VPN.

I would like to populate Assets and Identities in ES. Since Cloud instance cannot access the domain, the only way I can think of is using SA-LDAPSearch on Heavy Forwarder.

I set it up and successfully connects to LDAP.

Question:

How can I push the logs and create the lookup tables that will eventually populate the Assets and Identities in ES?

Thanks!

[richgalloway · Accepted solution]

Getting LDAP data into Splunk Cloud is not as straightforward as it is on-prem.  As you already know, one can't write LDAP data directly to a lookup file.  Well, you could, but that lookup file would be stuck on the HF where it does no good.

One method is to have the HF write the LDAP info to an index.  You can create a dedicated index with a short retention period for this.  A scheduled search runs on a Splunk Cloud SH to read the indexed data and write it to a lookup.

Another method is to use admon instead of LDAP.  Your AD forwarders would need to be configured to send admon events to the Cloud, but everything else would be done on the SH.

[b_chris21]

Hi @richgalloway,

thanks for your answer. I believe this guide is approximately what you described?

https://hurricanelabs.com/splunk-tutorials/your-guide-for-gathering-ldap-identity-data-with-splunk-c...

How can I write the |ldapsearch search results into an index? Is there an extra command for that?

I will use the expected asset and identity fields in ES (bunit, owner, priority etc).

Thanks again.

With kind regards, 

Chris

[richgalloway]

The instructions from Hurricane Labs uses Summary Indexing to write the ldapsearch results to an index.  Youc also can use the collect command.

[aruncp333]

1. Example search for collecting identity data from Active Directory:

|ldapsearch domain=<domain_name> search="(&(objectclass=user)(!(objectClass=computer)))" attrs="userAccountControl,sAMAccountName,personalTitle,displayName,givenName,sn,mail,telephoneNumber,mobile,manager,department,whenCreated,accountExpires"
|makemv userAccountControl
|search userAccountControl="NORMAL_ACCOUNT"
|eval suffix=""
|eval priority="medium"
|eval category="normal"
|eval watchlist="false"
|eval endDate=""
|table sAMAccountName,personalTitle,displayName,givenName,sn,suffix,mail,telephoneNumber,mobile,manager,priority,department,category,watchlist,whenCreated,endDate
|rename sAMAccountName as identity, personalTitle as prefix, displayName as nick, givenName as first, sn as last, mail as email, telephoneNumber as phone, mobile as phone2, manager as managedBy, department as bunit, whenCreated as startDate | collect <index-name>

This example search assigns static values for suffix, endDate, category, watchlist, and priority. Use it as a guide to construct and test a working search, then replace the static values with information from your AD environment. 

2. Example search for collecting asset data from Active Directory:

|ldapsearch domain=<domain name> search="(&(objectClass=computer))"
|eval city=""
|eval country=""
|eval priority="medium"
|eval category="normal"
|eval dns=dNSHostName
|eval owner=managedBy
|rex field=sAMAccountName mode=sed "s/\$//g"
|eval nt_host=sAMAccountName
|makemv delim="," dn
|rex field=dn "(OU|CN)\=(?<bunit>.+)"
|table ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_expected,should_timesync,should_update,requires_av  | collect <index-name>

3. On HF make sure of outputs.conf forward the data to splunk cloud.

4. ON SH run a scheduled search to create lookups