[imp] database backend for high-traffic sites?

[Liam Hoekenga]

We're currently using Oracle 8.1.7 as the database backend for our Horde/IMP
installation.  We have ~ 62.4k users in our userbase, 40k+ monthly (42.4k for
June 03), and 6000+/hr at peak usage (and those are just our summer stats..)

We've got 10 machines allocated to the pool (500mhz ultrasparc IIs w/ 2gb of
RAM, running Solaris 8).  We've recently switched all of them over to using the
SQL-based session handler (so they could be one multihomed vhost), and things
seem to be working ok if we limit the number of machines in the pool.
We're going to be replacing the pool of Solaris based Horde/IMP hosts with a
smaller pool of linux hosts (which we are hoping will perform better, based on
the hardware stats), but in the meantime, this is a fairly high profile service,
and it's important that it's fast and reliable.

With all 10 machines in the pool, our oracle server runs out of memory.
Unfortunately, it's maxed out @ 4GB (AXMP+, ultrasparc II, dual processor at
400mhz).  Sure, we could get a bigger, stompier oracle box, but we'd also prefer
not to continue having a single point of failure.

We'd like to pursue some DB load balancing, and we'd prefer to use something
that's built-in to the database platform (rather than hacking the horde code to
select the database based on condition /X/).  Oracle's "parallel server"
solution requires that all of the DB servers have a shared file system (like a
SAN). 

Does anyone here use MySQL or Postgres's replication to handle their databases?
Or perhaps a hardware load-balancer? How large is your installation?  

thx
Liam