[horde] Memcache server questions

[Michael M Slusarz]

Quoting Andrew Morgan <morgan at orst.edu>:

> On Sun, 16 Nov 2008, Michael M Slusarz wrote:
>
>> Quoting Kevin Konowalec <webadmin at ualberta.ca>:
>>
>>> Michael:
>>>
>>> How do you set memcache up to do that?  At the moment we have a  
>>> cluster of servers that horde points at but if any one of those  
>>> servers goes down the sessions stored on it are lost.  How can you  
>>> set it up so memcache servers are truly redundant (ie so sessions  
>>> are not lost if one machine fails)?
>>
>> If you want true, 100% fool-proof redundancy, then you will have to  
>> understand that memcache-only sessions is *NOT* designed to do  
>> this.  There are several hacks to improve the redundancy of the  
>> servers (see below), but they are all workarounds to the underlying  
>> issue that memcache was never meant to reliably store persistent  
>> data.  memcache was designed as a caching backend, and doing  
>> anything else on top of that is necessarily stretching what the  
>> original design decisions meant to accomplish.
>>
>> The implementation of the Horde memcache session handler has been  
>> based around several ideas:
>> 1.) A memcache server will rarely crash.  The memcache process  
>> itself is not a particularly complex, nor process/memory intensive  
>> process (most likely, an installation will run into issues with  
>> bandwidth limitations long before they will have issues with load  
>> on a server).  And a assumption I use when coding for large  
>> installations is that the memcache processes live on their own  
>> boxes (Obviously, that is not a requirement- I run a memcache  
>> process on the same server as the fastcgi processess for example).   
>> As such, redundancy is important, but not as important as  
>> performance.  This last portion ties in with number 2:
>> 2.) A session timeout due to a memcache failure is to be avoided,  
>> if possible, but is not a catastrophic event. Thus, trapping  
>> session failures events and provided a clean exit and proper error  
>> messages to the user is important. If running a cluster of memcache  
>> machines, failure of the server holding the session information  
>> will not take down the entire system - a user will be able to login  
>> immediately after a session failure with their session information  
>> being stored on one of the new
>>
>> That being said, the following two solutions can improve redundancy  
>> of the servers:
>> 1.) Use a SQL backend.  This will provide the best redundancy, with  
>> the downside of being more resource intensive (although we only  
>> write to the SQL backend if the session data has changed, and only  
>> after the page is displayed to the user).
>> 2.) Use the session handler.  The memcache module provides a  
>> session handler that is different than the Horde session handler,  
>> but may provide better redundancy.  Do note:
>> * The main functional difference between the two is that the  
>> "memcache" handler will not provide Horde session information,  
>> while the "Horde memcache" handler will
>> * I don't know if the "memcache" handler is smart enough to not  
>> write to the backend if the session data hasn't changed.  If not,  
>> the "Horde memcache" handler has a performance improvement here.
>> * If willing to run the beta version of the memcache module  
>> (version 3+; as opposed to the stable version 2 branch), there is  
>> an option to adding redundancy to session data if using "memcache"  
>> handler. This is something that is no currently possible with  
>> "Horde memcache" handler since the PECL API does not expose an  
>> option to determine how many servers to write a key to.  See, e.g.,  
>> http://pecl.php.net/bugs/bug.php?id=11637
>>
>> Obviously there is no way to know how any given installation has  
>> their backend setup, but I honestly can't think of a reason why you  
>> wouldn't want to combine all your memcache servers into a single  
>> pool (as opposed to breaking them up into smaller pools).  A site  
>> that has, for example, several boxes running lighttpd (or an  
>> equivalent) as a frontend, and multiple boxes running fastcgi and  
>> multiple boxes running memcached in the backend gives the prized  
>> goal of an installation where all elements are hotswappable (along  
>> with the advantage that a single configuration for all elements is  
>> needed).
>
> Sorry to revive this old thread...
>
> I'm looking to configure Horde3 to use a pair of memcache servers in  
> a redundant configuration as described above.  However, I am having  
> trouble translating Michael's comments to the Horde GUI setup options.
>
> Currently, I am using Memcache based sessions with the following config:
>
> $conf['sessionhandler']['type'] = 'memcache';
> $conf['sessionhandler']['memcache'] = true;
> $conf['memcache']['hostspec'] = array('memcache1.oregonstate.edu');
> $conf['memcache']['port'] = array('11211');
> $conf['memcache']['weight'] = array();
> $conf['memcache']['persistent'] = false;
> $conf['memcache']['c_threshold'] = 0;
> $conf['memcache']['compression'] = true;
> $conf['memcache']['prefix'] = 'hordetest';
> $conf['memcache']['large_items'] = true;
> $conf['memcache']['enabled'] = true;

$conf['memcache']['hostspec'] = array('memcache1.oregonstate.edu',  
'memcache2.oregonstate.edu');
$conf['memcache']['port'] = array(11211, 11211);

For redundancy, you will need memcache 3.0.0+.  You will have to set  
the php ini settings manually for these servers (see  
http://pecl.php.net/package-changelog.php?package=memcache&release=3.0.5)

michael

___________________________________
Michael Slusarz [slusarz at horde.org]