[dev] Horde_Ldap performance problem + patch

[LALOT Dominique]

2011/8/23 Xavier Montagutelli <xavier.montagutelli at unilim.fr>

> Hi list,
>
> I have identified a serious performance problem in the Horde_Ldap module.
>
> When doing a very simple search on a directory with around 50.000 entries,
> getting the results with the "shiftEntry" methods takes a *very* long time,
> compared to using the native PHP functions.
>
> This renders the "listUsers" method in the LDAP auth handler for Horde
> useless.
>
> Some numbers :
>
> 1. Using PHP native LDAP functions
> Number of entries                : 44588
> Search time                      : 2 seconds
> Building list from search result : 1 seconds
>
> 2. Using Horde_Ldap object
> Number of entries                : 44588
> Search time                      : 2 seconds
> Building list from search result : 112 seconds
>
>
> With Xdebug, I have identified the bottleneck. Each call to "shiftEntry"
> calls
> "count", and this one calls "ldap_count_entries". I suppose this function
> goes
> through all the remaining results to give the count, so calling it
> inefficient.
>
> With the following patch, it turns to be *much* quicker (~ 6 seconds to
> build
> the result list).
>
> --- Horde/Ldap/Search.php.orig  2011-08-23 12:41:42.000000000 +0200
> +++ Horde/Ldap/Search.php       2011-08-23 12:45:15.000000000 +0200
> @@ -145,10 +145,6 @@
>      */
>     public function shiftEntry()
>     {
> -        if (!$this->count()) {
> -            return false;
> -        }
> -
>         if (is_null($this->_entry)) {
>             $this->_entry = @ldap_first_entry($this->_link,
> $this->_search);
>             $entry = Horde_Ldap_Entry::createConnected($this->_ldap, $this-
> >_entry);
>
>
> >From my point of view, the call to "count" is useless, as ldap_first_entry
> or
> ldap_next_entry will return false at the end. Or do I miss something ?
>
>
> (BTW, the Net_LDAP2 code suffers from the same problem ...)
>
>
> My code to compare performance is :
>
>
> // Initialisation code
> [...]
> $filter = '(objectclass=inetOrgPerson)';
> $attr = array('uid');
>
> // Methode 1. Using LDAP native functions
> $t0 = time();
> $ldap = ldap_connect($h);
> ldap_bind($ldap, $user, $pass);
> $search = ldap_search ($ldap,
>                $basedn,
>                $filter,
>                $attr);
> $t1 = time();
>
> $userlist = array();
> $entry = ldap_first_entry($ldap, $search);
> while ($entry) {
>   $values = ldap_get_values ($ldap, $entry, 'uid');
>   $userlist[] = $values[0];
>   $entry = ldap_next_entry($ldap, $entry);
> }
> ldap_free_result($search);
> ldap_close($ldap);
> $t2 = time();
>
> echo "1. Using PHP native LDAP functions\n";
> echo "Number of entries                : " . count($userlist) . "\n";
> echo "Search time                      : " . ($t1-$t0) . " seconds\n";
> echo "Building list from search result : " . ($t2-$t1) . " seconds\n";
>
> // Methode 2. Using Horde_Ldap object
> $t0 = time();
> $ldap = new Horde_Ldap(array (
>                'hostspec' => $h,
>                'binddn' => $user,
>                'bindpw' => $pass));
> $search = $ldap->search(
>    $basedn,
>    $filter,
>    array ('attributes' => $attr)
> );
> $t1 = time();
>
> $userlist = array();
> while ($entry = $search->shiftEntry()) {
>    $userlist[] = $entry->getValue('uid', 'single');
> }
> $t2 = time();
>
> echo "2. Using Horde_Ldap object\n";
> echo 'Number of entries                : ' . count($userlist) . "\n";
> echo 'Search time                      : ' . ($t1-$t0) . " seconds\n";
> echo 'Building list from search result : ' . ($t2-$t1) . " seconds\n";
>
>
>
> --
> Xavier Montagutelli                      Tel : +33 (0)5 55 45 77 20
> Service Commun Informatique              Fax : +33 (0)5 55 45 75 95
> Universite de Limoges
> 123, avenue Albert Thomas
> 87060 Limoges cedex
> --
> Horde developers mailing list
> Frequently Asked Questions: http://horde.org/faq/
> To unsubscribe, mail: dev-unsubscribe at lists.horde.org
>


That's what I observed in a previous thread that leads to something looking
like a DOS for Apache. It was just listusers scanning a large directory.
Thanks Xavier for the debugging. By the way, we noticed that there is no
cache for that list. Is there a reason not to cache the results? A directory
is not changing so often..

Dom

-- 
Dominique LALOT
Ingénieur Systèmes et Réseaux
http://annuaire.univmed.fr/showuser.php?uid=lalot