[horde] Horde 3.1, memcache sessionhandler, sidebar, and flock()

[Chris Stromsoe]

On Fri, 29 Dec 2006, Chuck Hagenbuch wrote:
> Quoting Chris Stromsoe <cbs at cts.ucla.edu>:
>
>> _buildTree() in kronolith/lib/Block/tree_alarms.php is hanging at
>>
>>   require_once KRONOLITH_BASE . '/lib/base.php';
>>
>> base.php tries to create a calendar if there isn't a calendar for
>> that person already:
>
> Well, the good news for you is that we're moving towards getting shares 
> out of the DataTree for Horde 3.2. If you need shared calendars in the 
> interim, I suggest you search through the list archives for the various 
> ways that people have sped datatree stuff up. Otherwise, you can just 
> turn the datatree off and go without shares for now.

I need the shares.

I'm already using some of the DataTree speed-ups.  Enough so that I'm not 
having any database access issues; the slowest queries I'm seeing are < 
200ms to complete.

Normal calendar access is fast.  It's only the initial creation that's 
slow -- more than 5 than minutes to complete in some cases.  The call to 
GLOBALS['kronolith_shares']->exists(Auth::getAuth()) takes around 50 
seconds to complete, and the call to 
GLOBALS['kronolith_shares']->addShare($share) takes around 240 seconds to 
complete.

>From DataTree.php --

     function exists($object)
     {
         if (empty($object)) {
             return false;
         }
         if (is_a($object, 'DataTreeObject')) {
             $object = $object->getName();
         } elseif (is_array($object)) {
             $object = implode(':', $object);
         }

         if ($object == DATATREE_ROOT) {
             return true;
         }

// start
         $idMap = array_flip($this->_nameMap);
         if (isset($idMap[$object])) {
             return true;
         }

         $this->_load($object);

         $idMap = array_flip($this->_nameMap);
         return isset($idMap[$object]);
// end
     }


I replaced the block that runs from "start" to "end" with

         foreach ($this->_nameMap as $k => $v) {
             if ($object == $v) {
                 return true;
             }
         }
         $this->_load($object);

         foreach ($this->_nameMap as $k => $v)
             if ($object == $v) {
                 return true;
             }
         }
         return false;

and managed to cut 80s to 90s from the call (loading the sidebar dropped 
from ~300s to ~210s).


The call to "$GLOBALS['kronolith_shares']->addShare($share);" returns 
the results of "$this->_datatree->add($share);"

That calls exists() from DataTree.php, which calls _load() from 
DataTree/sql.php.  _load() runs the query

   SELECT c.datatree_id, c.datatree_name, c.datatree_parents,
          c.datatree_order
   FROM horde_datatree c
   WHERE c.group_uid = 'horde.shares.kronolith'

and then calls DB::getAll(), which loads the results by looping over all 
13,000+ of the entries.  I added time() calls into DB/common.php around 
the block that loops loading the rows, and I'm timing it at 175s to 
complete.


Is there any way to speed that up?  The postgres query planner shows the 
query running at 90ms if I run it by hand through pgsql.  Running the same 
query using native pg_* calls only returns all of the data in < 1s.


On a side-note, I'm still not getting mail from any of the Horde lists. 
Ticket #4814 has more information from somebody else who is seeing the 
same thing and a guess at why.



-Chris