[horde] Memcache issue

Fri May 23 08:09:41 UTC 2014

Citeren Arjen de Korte <arjen+horde at de-korte.org>:

> Citeren Administrator BCS <adje at bezoekerscentrumsonsbeek.nl>:
>
>> Quoting Arjen de Korte <arjen+horde at de-korte.org>:
>>
>>> Citeren Administrator BCS <adje at bezoekerscentrumsonsbeek.nl>:
>>>
>>>> Got Horde is up and running but performance is not what it should  
>>>> be. It seems I have a problem with memcache. Memcache is up and  
>>>> running, I can connect to it with telnet localhost 11211, but the  
>>>> stats tells me nothing is happening there.
>>>
>>> First question, how many concurrent users do you have? If there is  
>>> a performance issue, installing memcache is not magically going to  
>>> fix that for you. In you case it might help only if you have a  
>>> problem with storing/retrieving session information, but that will  
>>> happen only if you have many concurrent users and are starved on  
>>> I/O. And even then using native memcache sessions might be  
>>> preferred over using a distributed hashtable for caching session  
>>> information. Enabling the latter may even worsen the problem.
>>
>> I have only a few users, max 25 but in most cases only 3 or 4 are  
>> active simultaneously. The server is a brand new 3.2Ghz Xeon  
>> quadcore machine with 4Gb RAM. Apart from the  
>> Horde/apache2/postfix/dovecot/clamav stuff it's only running samba  
>> (only a few users as well). So it basically has nothing to do,  
>> processor load is a few % at max during peak moments.
>
> That system should be screaming with such a minimal load, so there  
> probably is indeed a configuration problem. I'm running Horde for a  
> similar load on an Intel NUC with Core i3-4010U processor and 8 GB  
> RAM, with subsecond response times.
>
> First of all, I assume you have not followed the steps from  
> http://www.horde.org/apps/horde/docs/PERFORMANCE yet. At the very  
> least, install an opcode cache (PHP accelerator) if you not already  
> did so. I'm using APC, both for opcode cache and as Horde Cache.  
> You'll need to allow 64-128 MB RAM for this.
>
>>>> When I enable Memcache server under  
>>>> Configuration>Horde>Distributed Hash Table, my conf.php contains:
>>>>
>>>> $conf['sessionhandler']['type'] = 'Builtin';
>>>> $conf['sessionhandler']['hashtable'] = true;
>>>> ...
>>>> $conf['hashtable']['params']['hostspec'] = array('localhost');
>>>> $conf['hashtable']['params']['port'] = array('11211');
>>>> $conf['hashtable']['params']['weight'] = array();
>>>> $conf['hashtable']['params']['persistent'] = false;
>>>> $conf['hashtable']['params']['compression'] = false;
>>>> $conf['hashtable']['params']['large_items'] = true;
>>>> $conf['hashtable']['driver'] = 'Memcache';
>>>
>>> The above all looks good. Assuming that 'horde-memcache-stats'  
>>> provide some sensible output, you can rule out that this is the  
>>> problem.
>>
>> Output from horde-memcache-stats:
>> horde-memcache-stats
>>
>> Server: localhost:11211 (Version: 1.4.13 - 4 thread(s))
>>    Size:          0,00 MB (Max: 64,00 MB - 0% used)
>>    Items:         0 (Total: 0)
>>    Cache Ratio:   0 hits, 0 misses
>>    Connections:   5
>>    Traffic:       0,00 MB in, 0,00 MB out
>>
>> It will stay that way all day.
>
> You're not running in a cluster, so memcache is not really needed  
> here. Even with SQL based session storage, you should not have  
> performance issues in storing sessions.

Thanks for this explanation, that was not clear to me so far. Turned  
memcache off in Horde, since it caused errors I will leave it like that.

>
>>>> As soon as I enable memcache this way, I see some "Invalid  
>>>> application" errors in Horde and I get randomly "User is not  
>>>> authorized for horde" messages in the Horde log, followed by  
>>>> logout. No other error messages. I set memcached logging to -vv  
>>>> and see no activity at all.
>>>>
>>>> I suppose I do something wrong in the Horde config but after a  
>>>> lot of reading and searching I got a bit lost now. Any pointers  
>>>> in the right direction welcome!
>>>
>>> What kind of performance issue do you have? And you do use an  
>>> opcode cache, right?
>>
>> Performance is more or less acceptable, but not the response speed  
>> I would expect based upon # of users, server load and server  
>> hardware. It can take over 5 seconds to process a user login and  
>> switching between let's say email and calendar also can take  
>> several seconds.
>
> A couple of suggestions:
>
> 1) Make sure Dovecot is using indexes and Postfix (in postfix/main.cf) uses
>
>     mailbox_command = /usr/lib/dovecot/deliver
>
> so that the IMAP indexes are updated upon delivery of mail (and not  
> when you check the mailboxes).

Had that configured already yes

> 2) Use caching for IMAP (imp/config/backends.local.php)
>
>    $servers['imap']['disabled'] = true;
>    $servers['advanced']['disabled'] = false;
>    $servers['advanced']['cache'] = 'sql';
>
> which will speed up IMP significantly.

Did so and noticed some improvement indeed, mail folders load quicker

>
> 3) Reduce checking for new mail (imp/config/prefs.local.php)
>
>    $_prefs['newmail_notify']['locked'] = true;
>    $_prefs['newmail_notify']['value'] = 0;
>
> so that not on every change of view checks for new messages run.

afraid our users won't like that...

>
> 4) Make sure your MySQL server is tuned for performance. I can't  
> give any specific advice here. Running it on a UNIX domain socket  
> will be faster than using TCP and you'll want to make sure that it  
> has enough cache available. Check with phpMyAdmin for instance if it  
> has suggestions for speeding things up.
>

I do run mysql on a socket. I checked before using mysqltuner and  
found all ok except some fragmentation, so I defragmented the tables.

I now installed phpmyadmin, great tool. I see a few red things showing  
up, will dive into that and tweak things where needed.

> 5) Check if you have enough Apache processes running to handle the load.
checked and increased a little.

>
> 6) Just 4 GB of RAM is not a lot for a quad core system. If all else  
> fails, consider adding more.
good point, ram is cheap so why not

>
>> Btw all users are on a direct wired lan connection so that should  
>> not be a bottleneck.
>>
>> About opcode caching: I have php5-memcache installed and phpinfo  
>> shows it is enabled.
>
> Memcache is not an opcode cache. You'll want something like APC (I  
> recommend this one), eAccelerator, or XCache. Before doing anything  
> else, this is the first thing you should do, as it will speed up PHP  
> dramatically without much fuss.
>

I did not realize that. I now installed APC, set it up for 128Mb and  
it makes a big difference indeed. So far hitrate is over 97% and 128Mb  
cache size seems just right.

All changes together seems to make a big difference, will see in the  
near future how things work out. Thanks all for the clarification and  
your suggestions!

>> Hope this helps you pointing out the problem. Could very well be  
>> that I am looking in the wrong place, I am by no means an expert.
>>
>>>
>>>> Running latest Horde Groupware 5.1.4 on Ubuntu Server 12.04
>>>>
>>>>
>>>> -- 
>>>> Horde mailing list
>>>> Frequently Asked Questions: http://horde.org/faq/
>>>> To unsubscribe, mail: horde-unsubscribe at lists.horde.org
>>>
>>>
>>> -- 
>>> This message was sent from a mailinglist subscription address.
>>> For off-list replies, you must remove the address extension.
>>
>>
>> -- 
>> Horde mailing list
>> Frequently Asked Questions: http://horde.org/faq/
>> To unsubscribe, mail: horde-unsubscribe at lists.horde.org
>
>
> -- 
> This message was sent from a mailinglist subscription address.
> For off-list replies, you must remove the address extension.