[horde] Memcache server questions

[Kevin Konowalec]

Thanks for the info Michael!

I'm hoping the patches you provided earlier will address our  
disappearing session problem... I'm getting a fair bit of heat about  
that one.

When using servers with large amounts of RAM is it better to run large  
instances of memcached or several smaller instances on each box?

K



On Nov 16, 2008, at 6:24 PM, Michael M Slusarz wrote:

> Quoting Kevin Konowalec <webadmin at ualberta.ca>:
>
>> Michael:
>>
>> How do you set memcache up to do that?  At the moment we have a  
>> cluster of servers that horde points at but if any one of those  
>> servers goes down the sessions stored on it are lost.  How can you  
>> set it up so memcache servers are truly redundant (ie so sessions  
>> are not lost if one machine fails)?
>
> If you want true, 100% fool-proof redundancy, then you will have to  
> understand that memcache-only sessions is *NOT* designed to do  
> this.  There are several hacks to improve the redundancy of the  
> servers (see below), but they are all workarounds to the underlying  
> issue that memcache was never meant to reliably store persistent  
> data.  memcache was designed as a caching backend, and doing  
> anything else on top of that is necessarily stretching what the  
> original design decisions meant to accomplish.
>
> The implementation of the Horde memcache session handler has been  
> based around several ideas:
> 1.) A memcache server will rarely crash.  The memcache process  
> itself is not a particularly complex, nor process/memory intensive  
> process (most likely, an installation will run into issues with  
> bandwidth limitations long before they will have issues with load on  
> a server).  And a assumption I use when coding for large  
> installations is that the memcache processes live on their own boxes  
> (Obviously, that is not a requirement- I run a memcache process on  
> the same server as the fastcgi processess for example).  As such,  
> redundancy is important, but not as important as performance.  This  
> last portion ties in with number 2:
> 2.) A session timeout due to a memcache failure is to be avoided, if  
> possible, but is not a catastrophic event. Thus, trapping session  
> failures events and provided a clean exit and proper error messages  
> to the user is important. If running a cluster of memcache machines,  
> failure of the server holding the session information will not take  
> down the entire system - a user will be able to login immediately  
> after a session failure with their session information being stored  
> on one of the new
>
> That being said, the following two solutions can improve redundancy  
> of the servers:
> 1.) Use a SQL backend.  This will provide the best redundancy, with  
> the downside of being more resource intensive (although we only  
> write to the SQL backend if the session data has changed, and only  
> after the page is displayed to the user).
> 2.) Use the session handler.  The memcache module provides a session  
> handler that is different than the Horde session handler, but may  
> provide better redundancy.  Do note:
>  * The main functional difference between the two is that the  
> "memcache" handler will not provide Horde session information, while  
> the "Horde memcache" handler will
>  * I don't know if the "memcache" handler is smart enough to not  
> write to the backend if the session data hasn't changed.  If not,  
> the "Horde memcache" handler has a performance improvement here.
>  * If willing to run the beta version of the memcache module  
> (version 3+; as opposed to the stable version 2 branch), there is an  
> option to adding redundancy to session data if using "memcache"  
> handler. This is something that is no currently possible with "Horde  
> memcache" handler since the PECL API does not expose an option to  
> determine how many servers to write a key to.  See, e.g., http://pecl.php.net/bugs/bug.php?id=11637
>
> Obviously there is no way to know how any given installation has  
> their backend setup, but I honestly can't think of a reason why you  
> wouldn't want to combine all your memcache servers into a single  
> pool (as opposed to breaking them up into smaller pools).  A site  
> that has, for example, several boxes running lighttpd (or an  
> equivalent) as a frontend, and multiple boxes running fastcgi and  
> multiple boxes running memcached in the backend gives the prized  
> goal of an installation where all elements are hotswappable (along  
> with the advantage that a single configuration for all elements is  
> needed).
>
> michael
>
> -- 
> ___________________________________
> Michael Slusarz [slusarz at horde.org]
>
>