[imp] Horde/IMP performance and system requirements (50k+ accounts)

[Jie Gao]

* Janne Peltonen <janne.peltonen at helsinki.fi> wrote:

> Hi!
> 
> We've been running IMP here at the University of Helsinki since 2001,
> and performed a major upgrade last summer, straight into Horde Webmail
> Edition 1.1.1. Since then, there has been no end of problems we've been
> running into, most notably all kinds of performance problems
> - so much so that we've been considering moving out of Horde/IMP
> altogether. At least my boss is getting convinved that Horde just
> doesn't work.
 
We have 80k user accounts here. We tried to upgrade from 2.* to 3.*
a few years ago and had to rollback due to a performance problem caused
by a bad sql query in the code; adding more resources did not help.

We tried again with Horde Webmail Edition a few weeks ago, and we had
plenty of resources ready: 6 front-end redhat servers (in vmware - not
my preference though), and a dual Quad-Core AMD Opteron redhat server
(16GB mem) for the mysql database (it is used as a front-end server as
well now).

We immediately encountered performance problem, and found that it was
caused by dynamic address matching in the compose window, and as it
would match almost everything when you type in the first letter of an
email address, our ldap server was brought to its knees!

We disabled the use of ldap immediately. 

I guess it is doing the same thing with everybody's addressbook in 
the mysql database as well, but the db seems to cope.

Yes, we monitor and tune the mysql server all the time, which is handling
2175 queries per second on average.

We seem to be ok now, and are only seeing problems with wvware and xhtml
looping out of control, which we have a running watchdog to kill.

Another problem is user getting a blank screen when viewing messages,
and we are still trying to find out why: seems to be caused by user
using a Chinese-language OS.

Hope this helps.

Regards,



Jie

> But to concentrate on performance. The old IMP hopped along nicely on
> three load balanced x86 servers, with 2-4 cores at 2,3 GHz and 2-4 GB
> RAM, running Linux - the load was negligible (actually, the server with
> 4 cores and 4GB could handle all the load if need be). And the load on
> the database server was negligible. Now, we have three xen virtual
> servers, each with 4 cores @ 2,3 GHz (64 bit) and 6 GB memory, and they
> tend to choke at peak load - during the day, the load is always
> noticeable, it takes a couple seconds for each request to complete.
> Also, the mysql server tends to run with the mysqld process occupying
> one processor completely and hogging all the memory it can get (2GB, the
> server running on a 32 bit system). We had to disable Kronolith, Nag and
> Mnemo completely, because they just completely choked the mysql server.
> 
> We are using all kinds of performance tweaks we could find information
> on, we are using a php cache (apc) and so on, but still the problems
> persist... So I was wondering if there are other people using a new
> Horde Webmail Edition in a similar setting: a university, ~50k accounts
> (including students & staff), peak concurrent users ~5000, peak
> logins/minute ~160. If anyone is, we'd like to hear about your
> performance experiences and also, what kind of hardware / OS you are
> running. Also, if there were any specific tricks that helped you reduce
> the server load, we'd love to hear about them.
> 
> Thanks for any answers.
> 
> 
> --Janne Peltonen
> University of Helsinki
> -- 
> Janne Peltonen <janne.peltonen at helsinki.fi> PGP Key ID: 0x9CFAC88B
> Please consider membership of the Hospitality Club (http://www.hospitalityclub.org)
> --
> IMP mailing list - Join the hunt: http://horde.org/bounties/#imp
> Frequently Asked Questions: http://horde.org/faq/
> To unsubscribe, mail: imp-unsubscribe at lists.horde.org