[horde] Memcache server questions

[Michael M Slusarz]

Quoting Kevin Konowalec <webadmin at ualberta.ca>:

> Michael:
>
> How do you set memcache up to do that?  At the moment we have a  
> cluster of servers that horde points at but if any one of those  
> servers goes down the sessions stored on it are lost.  How can you  
> set it up so memcache servers are truly redundant (ie so sessions  
> are not lost if one machine fails)?

If you want true, 100% fool-proof redundancy, then you will have to  
understand that memcache-only sessions is *NOT* designed to do this.   
There are several hacks to improve the redundancy of the servers (see  
below), but they are all workarounds to the underlying issue that  
memcache was never meant to reliably store persistent data.  memcache  
was designed as a caching backend, and doing anything else on top of  
that is necessarily stretching what the original design decisions  
meant to accomplish.

The implementation of the Horde memcache session handler has been  
based around several ideas:
1.) A memcache server will rarely crash.  The memcache process itself  
is not a particularly complex, nor process/memory intensive process  
(most likely, an installation will run into issues with bandwidth  
limitations long before they will have issues with load on a server).   
And a assumption I use when coding for large installations is that the  
memcache processes live on their own boxes (Obviously, that is not a  
requirement- I run a memcache process on the same server as the  
fastcgi processess for example).  As such, redundancy is important,  
but not as important as performance.  This last portion ties in with  
number 2:
2.) A session timeout due to a memcache failure is to be avoided, if  
possible, but is not a catastrophic event. Thus, trapping session  
failures events and provided a clean exit and proper error messages to  
the user is important. If running a cluster of memcache machines,  
failure of the server holding the session information will not take  
down the entire system - a user will be able to login immediately  
after a session failure with their session information being stored on  
one of the new

That being said, the following two solutions can improve redundancy of  
the servers:
1.) Use a SQL backend.  This will provide the best redundancy, with  
the downside of being more resource intensive (although we only write  
to the SQL backend if the session data has changed, and only after the  
page is displayed to the user).
2.) Use the session handler.  The memcache module provides a session  
handler that is different than the Horde session handler, but may  
provide better redundancy.  Do note:
   * The main functional difference between the two is that the  
"memcache" handler will not provide Horde session information, while  
the "Horde memcache" handler will
   * I don't know if the "memcache" handler is smart enough to not  
write to the backend if the session data hasn't changed.  If not, the  
"Horde memcache" handler has a performance improvement here.
   * If willing to run the beta version of the memcache module  
(version 3+; as opposed to the stable version 2 branch), there is an  
option to adding redundancy to session data if using "memcache"  
handler. This is something that is no currently possible with "Horde  
memcache" handler since the PECL API does not expose an option to  
determine how many servers to write a key to.  See, e.g.,  
http://pecl.php.net/bugs/bug.php?id=11637

Obviously there is no way to know how any given installation has their  
backend setup, but I honestly can't think of a reason why you wouldn't  
want to combine all your memcache servers into a single pool (as  
opposed to breaking them up into smaller pools).  A site that has, for  
example, several boxes running lighttpd (or an equivalent) as a  
frontend, and multiple boxes running fastcgi and multiple boxes  
running memcached in the backend gives the prized goal of an  
installation where all elements are hotswappable (along with the  
advantage that a single configuration for all elements is needed).

michael

-- 
___________________________________
Michael Slusarz [slusarz at horde.org]