Security

AD is only returning 1000 entries. How do I get around this?

the_wolverine
Champion

I'm trying to configure LDAP auth for Splunk. I'm running into an issue where AD is only giving me 1000 entries and the users I'm looking for are not being returned. How do I get around this?

Tags (2)
1 Solution

the_wolverine
Champion

By default, AD imposes a sizelimit of 1000 entries. This is the max number of entries that can be returned in a single bind request. There is no way to modify this (outside of AD) so we must use userBaseFilters and/or groupBaseFilters to reduce the number of entries returned.

A separate call is made on userBaseDN and groupBaseDN so the max is really 1000 user entries and 1000 group entries PER DN. If you specify multiple DNs in your configuration, multiple calls are made to AD, each which allows the max sizelimit.

A workaround?

  • Try using a more specific BaseDN. For example, instead of dc=SplunkSupport,dc=Com, use a combination of : ou=People,ou=USA,dc=SplunkSupport,dc=Com;ou=People,ou=Canada,dc=SplunkSupport,dc=Com;

  • Use LDAP filters. See this blog post for examples of LDAP filters used to further reduce the number of entries returned.

  • Use the AD GC port.

View solution in original post

matt_1
Explorer

I actually ended up having to specify each user's sAMAccountName within the "userBaseFilter". Fortunately I was able to write a script to generate groupBaseDN, groupBaseFilter, userBaseDN, and userBaseFilter by wrapping the Solaris ldapsearch.

The goal is to automate that portion so that Splunk access can be inherited through the AD Group, distribution list, or mailbox member.

Here is the pilot script. The final one should actually build the authentication.conf, reload auth, set user defaults relative to roles (i.e. set the default application relative to user and role), and be managed through a configuration file. Use Splunk cron scheduler to run once or twice a week.

#!/usr/bin/perl

my(@domains) = (
  '<domain label 1> ',
  '<domain label 2> ',
  '<domain label n> ',
);

my(%cfgs);
if(@ARGV)
{
  push(@{$cfgs{adGroups}},@ARGV);
}
else
{
  @{$cfgs{adGroups}} = (
    '<AD Group sAMAccountName 1>',
    '<AD Group sAMAccountName 2>',
    '<AD Group sAMAccountName n>',
  );
}

%{$cfgs{adBindParameters}} = (
  '<domain label 1>' => {
    'cn' => 'CN=<...>,OU=<...>,DC=<....>,DC=com',
    'pw' => '<Bind Password>',
    'dc' => '<Domain Controller or LDAP Host>',
    'pt' => '<port>',
    'tr' => '<Top OU of Search Path>'
  },
  '<domain label 2>' => {
    ...........
  }
  '<domain label n>' => {
    ...........
  }
);

my($regex) = '^CN=[^,]+,[^,]+,\s*?(OU\=[^$]+)$';
foreach my $domain (@domains)
{
  print "##\n## [[$domain]]----------------------------\n##\n";
  my(@userBaseDN);
  my(@groupBaseDN);
  my($groupList);
  my($userBaseFilter) = '(|';
  my($groupBaseFilter) = '(|';
  foreach my $group (@{$cfgs{adGroups}})
  {
    my($memberList);
    my($cmd) = (
      '/usr/bin/ldapsearch' .
      ' -v' .
      ' -x' .
      ' -h "' . $cfgs{adBindParameters}{$domain}{dc} . '"' .
      ' -p "' . $cfgs{adBindParameters}{$domain}{pt} . '"' .
      ' -b "' . $cfgs{adBindParameters}{$domain}{tr} . '"' .
      ' -D "' . $cfgs{adBindParameters}{$domain}{cn} . '"' .
      ' -w "' . $cfgs{adBindParameters}{$domain}{pw} . '"' .
      ' "(sAMAccountName=' . $group . ')"' .
      ' -W "member"'
    );

    #--------------------------------------------------------------#
    # Grab DN for each group member and DN for evaluated AD group. #
    #--------------------------------------------------------------#
    if(open(adInfo,"$cmd|"))
    {
      my($memberJoinFlag);
      my($groupJoinFlag);
      while(<adInfo>)
      {
        s/^\s+|\s+$//g;
        s/\\/\\\\\\\\/gm;
        if(/^member:\s*(.*)/)
        {
          $memberJoinFlag = 1;
          $memberList .= "\n$1";
        }
        elsif(/^dn:\s*(.*)/)
        {
          $groupJoinFlag = 1;
          $groupList .= "\n$1";
        }
        elsif($memberJoinFlag && !/^[\d]+\smatches/)
        {
          $memberJoinFlag;
          $memberList .= $_;
        }
        elsif($groupJoinFlag && !/^[\d]+\smatches/)
        {
          $groupJoinFlag;
          $groupList .= $_;
        }
      }
      close(adInfo);
    }

    #---------------------------------------------------------------#
    # Determine each member's UID (sAMAAccountName) from DN.  Build #
    # 'userBaseFilter and 'groupBaseFilter'                         #
    #---------------------------------------------------------------#
    my(@memberDNs) = split("\n",$memberList);
    my($duplicate);
    foreach my $member (@memberDNs)
    {
      if($member =~ /$regex/)
      {
        my($ou) = $1;
        my($dupeKey) = $ou;
        $dupeKey =~ s/[\W]//gm;

        unless(defined $duplicate{$dupeKey})
        {
          push(@userBaseDN,"$ou");
        }
        $duplicate{$dupeKey} = 1;
      }

      $cmd = (
        '/usr/bin/ldapsearch' .
        ' -v' .
        ' -x' .
        ' -h "' . $cfgs{adBindParameters}{$domain}{dc} . '"' .
        ' -p "' . $cfgs{adBindParameters}{$domain}{pt} . '"' .
        ' -b "' . $cfgs{adBindParameters}{$domain}{tr} . '"' .
        ' -D "' . $cfgs{adBindParameters}{$domain}{cn} . '"' .
        ' -w "' . $cfgs{adBindParameters}{$domain}{pw} . '"' .
        ' "(distinguishedName=' . $member . ')"' .
        ' -W "sAMAccountName"'
      );
      if(open(adInfo,"$cmd|"))
      {
        while(<adInfo>)
        {
          s/^\s+|\s+$//g;
          if(/^sAMAccountName:\s*([\w]+)$/)
          {
            my($userAcctName) = uc($1);
            if(
              ($userAcctName =~ /^[aAuU][aA-zZ,0-9]{6}$/) &&
              (!defined $duplicate{$userAcctName})
            )
            {
              $userBaseFilter .= "(sAMAccountName=$userAcctName)";
            }

            unless(defined $duplicate{$group})
            {
              $groupBaseFilter .= "(sAMAccountName=$group)";
            }

            $duplicate{$userAcctName} = 1;
            $duplicate{$group} = 1;
          }
        }
        close(adInfo);
      }
    }
  }
  $groupBaseFilter .= ')';
  $userBaseFilter  .= ')';

  #-----------------------------------------------------------#
  # Determine a list of all OU search paths for "groupBaseDN" #
  #-----------------------------------------------------------#
  my(@groupDNs) = split("\n",$groupList);
  my(%duplicate);
  foreach(@groupDNs)
  {
    if(/$regex/)
    {
      my($ou) = $1;
      my($dupeKey) = $ou;
      $dupeKey =~ s/[\W]//gm;

      unless(defined $duplicate{$dupeKey})
      {
        push(@groupBaseDN,"$ou");
      }
      $duplicate{$dupeKey} = 1;
    }
  }

  print "groupBaseDN = " . join(';',@groupBaseDN) . "\n\n";
  print "groupBaseFilter = $groupBaseFilter\n\n";
  print "userBaseDN = "  . join(';',@userBaseDN) . "\n\n";
  print "userBaseFilter = $userBaseFilter\n\n";
}


__END__
0 Karma

the_wolverine
Champion

By default, AD imposes a sizelimit of 1000 entries. This is the max number of entries that can be returned in a single bind request. There is no way to modify this (outside of AD) so we must use userBaseFilters and/or groupBaseFilters to reduce the number of entries returned.

A separate call is made on userBaseDN and groupBaseDN so the max is really 1000 user entries and 1000 group entries PER DN. If you specify multiple DNs in your configuration, multiple calls are made to AD, each which allows the max sizelimit.

A workaround?

  • Try using a more specific BaseDN. For example, instead of dc=SplunkSupport,dc=Com, use a combination of : ou=People,ou=USA,dc=SplunkSupport,dc=Com;ou=People,ou=Canada,dc=SplunkSupport,dc=Com;

  • Use LDAP filters. See this blog post for examples of LDAP filters used to further reduce the number of entries returned.

  • Use the AD GC port.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...