[horde] Re: High Capacity Horde & Email Environment

[Etienne Goyer]

On Wed, Mar 31, 2004 at 11:26:50AM -0600, Jacob Davida wrote:
> >We do large scale (70K accounts last time I checked) Horde with Cyrus
> >Murder.  We present a single image to client by using LVS.  Horde
> >preference are kept in a single database that is accessed by each
> >frontend (two of them).  So far, Horde have been trouble free in this
> >setup.  We had some major problem with the Murder, but nothing someone
> >careful could not avoid.
> 
> Sounds great. What MTA do you use? Where are your users stored?

Postfix.  Each frontend (two of them run one).  Authentication is done
against an AD with Kerberos (Cyrus -> saslauthd -> Kerberos -> AD), but
we could have used LDAP too.  Alias resolution is done against the AD
via LDAP.
 
> >So far, the performance exceed my expectation.  I thought the frontends
> >would melt from being used by so many accounts, but so far they are
> >holding thight serving IMAP/POP, Horde/IMP and doing MTA duties.
> 
> Good to hear. What hardware are you using for each piece?

Frontend are dual Xeon, 4 GB RAM.  Load barely ever get 5 there.
Backend are single Xeon, 2 GB RAM with hardware SCSI RAID (don't know
which) and fast disk.  Load barely ever get over one there.  The MUPDATE
master also host the MySQL database used from Horde pref on a machine
similar to backend minus the disk array.  Load alaways below 1 there
too, but I had to do a little tweaking of MySQL.

> Thanks for the input. Would it be possible to see a conceptual diagram of
> or even explanation of how everythign is interacting? What OS(s) do you
> use, etc. Feel free to respond offline if you feel this is inappropriate for
> a discussion board. jacob-news at davida dot com

Well, I don't have a diagram, but it's really quite simple.  We have
redundant LVS machine splitting the incoming load on the two frontends.
The frontend run Cyrus POP and IMAP proxy daemon, an MTA (Postfix) 
delivering to a local LMTP proxy and Horde.

Horde is the simplest component.  It is a standalone installation that
store its prefs in a MySQL database on another machine (the MUPDATE
master; more on that later) so users can connect on either frontend.
We setup connection affinity on the LVS for https (how user connect to
Horde) because the session information is not shared between frontend,
but I guess we could get without this if we worked up some kind of
shared session for PHP.

Regarding the Murder (everything Cyrus-related), all the process running
on the frontends are proxy.  They resolve the real mailbox location via
the MUPDATE protocol against the MUPDATE master, then carry on the
operation on the correct backend on behalf of the client.  This is a
somewhat complex matter so I suggest you read the Cyrus Murder
documentation.

Concerning Postfix, the mail is delivered to a local LMTP proxy, which
resolve the destination backend via MUPDATE and carry on the delivery on
that machine.  No mail is delivered locally on the frontend.

> What happens when you need to add more storage space? 

Good question.  Cyrus have the concept of mail spool partition, which
reside in different directory.  You could add more space by mounting
more disks and making a mail spool partition there.  You can also move
mailbox between partition.  Please keep in mind that the only machine
hosting mailbox are the backends.

> or more performance
> on the IMAP/POP servers? what about for the MTA servers?

Just add another frontend or three !  This is really quite simple as the
frontend are totally stateless.  The scalability in this regard is
infinite (well, almost).

> Or maybe this isn't an issue. Is each machine running all services, and then
> accessing
> one central data storage location?

No, each backend host his own set of mailbox.  A mailbox reside on a
single backend, nothing is mirrored.  The MUPDATE master and frontend
are stateless and don't share any kind of storage (except for th Horde
prefs being host on a shared MySQL database).

> everyone so far. LVS looks to be a promising item to be included in this
> setup.

Indeed.  It is not really intuitive at first, but we had 0 problem with
it and the performance is fine.  I recommend it.

Regards,

-- 
Etienne Goyer                    Linux Québec Technologies Inc.
http://www.LinuxQuebec.com       etienne.goyer at linuxquebec.com