[horde] php and postgres, Re: Horde 3.1, memcache sessionhandler, sidebar, and flock() (fwd)

[Chris Stromsoe]

I sent this directly to Michael M Slusarz <slusarz at horde.org> instead of 
horde at lists.horde.org.  This is a re-send.

On Tue, 2 Jan 2007, Michael M Slusarz wrote:

> Also - seeing as this seems to be an issue with subsequent accesses (i.e. the 
> first exists() access, according to Chris' analysis, runs extremely fast - it 
> is multiple calls to exists() that really destroys performance), maybe this 
> is an issue with data structures/memory usage in PHP?  i.e. some of the 
> issues we have fixed in IMP lately dealing with looping code and memory 
> allocation such as removal of newlines in large messages, quoted-printable 
> encoding with large messages, and doing PHP splitting of large MIME messages. 
> Possibly the same issue?

I would guess that the problem is somewhere along those lines.  I only see the 
problem exists() for large tables, and only on the second and subsequent calls. 
The first call is always very fast.

I wasn't seeing the problem with mnemo shares, but there are only a handful of 
mnemo shares in my test setup.  I do see the problem with kronolith (about 13k 
shares), and with nag (around 20k shares -- the problem is much worse).


> Of course, I know pretty much zero about the datatree/shares system so this 
> may just be noise.

I've traced the performance problem to the PHP database calls, so it's probably 
not a datatree/shares problem.

For the first call to exists(), calls to pg_fetch_row() return in maybe 1/4 to 
1/3 of a microsecond (measured using microtime() -- 2 to 3 calls return with 
the same time-stamp).

For the second call to exists(), between 10 and 16 calls to pg_fetch_row() 
return with the same timing characteristics as the first call to exists(). All 
of the subsequent calls to pg_fetch_row() take ~100ms to return.



-Chris