[horde] horde/imp migration problems, cpu bound on solaris

[Eric Rostetter]

Quoting Vasilios Hoffman <vhoffman01 at wesleyan.edu>:

> compiled w/ gcc 3.2.2 (first gcc 2.95.3 and then again w/ 3.2.2) and it

Sun's compilers generally will result in better code than gcc in my
experience.

> has the bells and whistles (imap support, ldap support, hashed php
> sessions, imapproxy, whatever else I could find and throw at it that it
> seemed to want).

Try a php accelerator.  Also mcrypt in php.  Both of these were already
mentioned.  If you have lots of mail boxes (you don't say how many mailboxes
you serve) you might be able to do some kernel tuning to cache more inodes
and files in memory...

> Now, previously Horde/Imp was I/O bound -- those flat text inboxes scale
> poorly and every page reload would drag lots of I/O off the server.  Was

You can often tune this to use memory cache more efficiently...  But
obviously the mailbox format is the major consideration.

> The new Horde/Imp.... CPU bound.  Ridiculously so.  Bringing up the login
> page eats up 1% of the cpu

1% of one cpu I assume?  That is nothing really.

> and a single login takes up to 8% of the
> processor AND takes seconds to execute.

Are you sure the "seconds to execute" isn't really in the imap connection
and authentication, rather than in the php/horde software?

> Seconds.  Like 4 or 5, not 1.

My logins take a while too.  But there is a lot going on in a login, including
applying filters, checking mailboxes for the summary screen, getting any other
info (for me, stock quotes, weather, etc) for the summary screen, and so on).

So simply saying the login takes a long time doesn't really mean much.  It
could be a slow connection to the mail server, a slow authentication to the
mail server, other functions being done at login time, DNS or identd lookups
that are slow, firewalls slowing things down, etc.

> We tried a test on the machine, and about 35 concurrent connections would
> cripple the CPU.  I/O was fine, memory was fine, but the load jumped over
> 50 and the machine was rendered unusable.

That certainly is very bad, and I would not expect that from the class
of machine.

> I can't believe that php or the new Horde/Imp setup is unusable with 35
> connections on a 4-processor sun enterprise server.  I realize there is
> more php to process... but... even 1 user is too slow!!

You need to find out what is slow.  PHP is slow, and an accelerator will
help.  But I'd guess you have some slowness elsewhere as well.

> and it's definately php -- if I compile php as a DSO, I can watch libphp
> during a login in top and truss, and it's chugging along with 5-8% of the
> processor and no I/O.

That doesn't mean per se it is php.  It does help eliminate some other
causes.

> So is this normal?

No.

> Is this the performance everybody else is getting?

No.

> Should I be talking to the php folks, or is it something specific to the
> horde code?

It isn't Horde, unless you do things on the summary page and so on to slow
it down.

It could be php, it could be apache, it could be your mail server, it
could be a firewall, etc.  You've really not told us enough to know for
sure.

> Is there anything I can do to speed up the php processing?

See previous postings (in particular the one pointing to a performance
document).

> I looked at the php readme's,etc, and I've configured the pants off it and
> every other application involved and I don't think I missed anything
> obvious.

You really need to isolate where the major slowness is.  How long, for
example, does a page that doesn't use the imap server run compared to
one that does use the imap server?  That can help answer if the imap
server is part of the problem or not...  If you connect via a local
web browser on the loopback device is it slower/faster than a remote
browser over the network?  If so, your network may be slow...
Simple tests like that will go a long way to finding out where the
problems/slowness is really at.

> Any feedback would be appreciated.  If there's some easy php profiler I
> can use to find out where it's spinning it's wheels, that would be great
> as I'd rather not profile it by hand.  I'm hoping it's something I can
> fix, and not just all the new functionality slowing things down
> significantly.

See previous posts about profilers...

> If it's just the new code, than please tell me what kind of machine you
> all think could reasonably process say, ~100 active users

A lot less machine than you are using.

--
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Why get even? Get odd!