[imp] Imp Performance Problem

[Michael M Slusarz]

Quoting Registration <registration at portabile.net>:

>> On 2015-03-30, at 23:21 , Registration <registration at portabile.net> wrote:
>>
>>
>>> On 2014-10-29, at 17:45 , Registration <registration at portabile.net> wrote:
>>>
>>>
>>>> On 2015-03-29, at 14:10 , Arjen de Korte <arjen+horde at de-korte.org> wrote:
>>>>
>>>> Citeren Registration <registration at portabile.net>:
>>>>
>>>>> Dear Imp users and developers,
>>>>>
>>>>> my problem is as follows:
>>>>> - Login to Horde takes 3 seconds.
>>>>> - Loading mails takes 7 seconds for a larger IMAP folder, and  
>>>>> 1-2 seconds for one with only a few messages in it.
>>>>> - Top shows 97% CPU from php5-fpm, 3% from imap.
>>>>
>>>> That means that performance problems are probably in php. Do you  
>>>> use an php opcode cache / accelerator? If so, which one?
>>>
>>> XCache 2.0.0
>>>
>>>>
>>>>> - When messages are cached, everything gets super fast.
>>>>> - The loading times are probably proportional to the amount of  
>>>>> message headers Imp loads for viewing.
>>>>>
>>>>> My questions are:
>>>>> - Is this normal speed?
>>>>
>>>> It depends. How large is a large IMAP folder? How many folders do  
>>>> you have?
>>>
>>> I think that Imp caches something a few dozen messages around the  
>>> currently displayed ones.
>>> So the IMAP folder size is not important, just that there are more  
>>> than a few:
>>> For example, an inbox with 10 messages is loaded in a second, a  
>>> view of a folder with 900 messages
>>> takes 6-8 seconds, and also 6-8 seconds for a folder with 20000 messages.
>>>
>>>>
>>>>> - If not, how would I systematically search for the problem?
>>>>> - I tried xhprof, but I can't get the part where Imp refreshes  
>>>>> messages captured.
>>>>>
>>>>> My config:
>>>>> - Debian Wheezy, Apache, php5-fpm, Dovecot, imapproxy
>>>>
>>>> Without version numbers, there is no way for us to tell what  
>>>> you're actually using.
>>>
>>> Kernel 3.2.0-4-amd64, Apache 2.2.22, PHP 5.4.39-0+deb7u1, Zend  
>>> 2.4.0, XCache 2.0.0, Dovecot 2.1.7
>>>
>>>>
>>>>> - Auth using Imp using MySQL/PDO connection via sockets
>>>>> - Dovecot using Maildir, PAM disabled, mysql auth only
>>>>> - Horde caching enabled
>>>>
>>>> Which cache backend?
>>>>
>>>>> - everything on one host
>>>>
>>>> If everything is on the same host, imapproxy is redundant an may  
>>>> even hurt performance if you have a modern IMAP server like  
>>>> Dovecot.
>>>
>>> Ok, disconnected and purged it, no visible difference.
>>>
>>>>
>>>>> - standard rented server
>>>>
>>>> I wonder what the specifications of a 'standard rented server' are. :-)
>>>
>>> AMD Opteron 6272, 2 GB RAM, 100 GB SSD - not high end, but not  
>>> stone-age either, I think
>>>
>>>>
>>>>> - everything up-to-date
>>>>
>>>> What may be up-to-date on Debian Wheezy may still be seriously  
>>>> out-of-date.
>>>
>>> I know, but no testing builds for me anymore...
>>
>> I did a performance test using  
>> http://www.php-benchmark-script.com/bench.php, which resulted in ~7  
>> seconds for my server, same time as the test on the benchmarks  
>> webpage.
>>
>> The only thing I could find which could be related to this is the  
>> Horde-specific message:
>> FastCGI: server "/usr/lib/cgi-bin/php5-fcgi" stderr: PHP message:  
>> Cannot write Autoloader cache to backend.
>> I tried setting proper path permissions, but no success, the error  
>> doesn't go away.
>>
>
> I managed to get the folder view loading time down to four seconds  
> for a larger IMAP folder  (>1k files) like "sent" or "archive".
> Could anyone tell me how long it takes to display a folder on their  
> server? I don't have access to a second horde installation.

My 12,000 message Trash folder takes around ~2 seconds to load the  
initial view.

Understand that this kind of operation (message list) is almost  
certainly a bottleneck at the IMAP level, rather than the PHP level.   
Your performance is going to be pretty bad on large mailboxes unless  
using a caching IMAP server or caching on the Horde side (or both).

michael

___________________________________
Michael Slusarz [slusarz at horde.org]