[imp] login slowness with large inboxes after upgrade from Horde 3.0.10+IMP 4.0.4 to Horde 3.3.8 + IMP 4.3.7

Jose Manuel Blanco jmblanco at sciops.esa.int
Thu Nov 4 08:34:55 UTC 2010


Thank you all for your replies

Regarding the IMAP server side suggestions, the thing is that my old 
webmail (Horde 3.0.10+IMP 4.0.4) is connecting to the same IMAP server, 
and it has no problems dealing with large inboxes -1 or 2 seconds max. 
for displaying inboxes with more than 40000 mails never been cached-, so 
my question remains: has the imp code changed in a way that would 
explain that behaviour? For example, when I click in the Date tab to 
reorder my messages, my old webmail only reorders the ones that are 
visible, while the new one goes to the first (or last) page instead. 
This suggests that the old one is only dealing with the ones that are 
visible, but I can't be sure. The settings are the same in both webmails 
as far as I can see.

Thanks again for your help.

Jose Manuel


On 11/03/2010 07:12 PM, Michael M Slusarz wrote:
> Quoting Aria Bamdad <aria at bsc.gwu.edu>:
>
>> I have the same problem with my setup.  I use an IBM IMAP server and 
>> after
>> running several traces, we found that IMP tends to request specific 
>> detailed
>> information about every mail item in the inbox at logon.  When you have
>> 1000's of messages and if your IMAP server is not local or is not super
>> fast, this results in the slow login problem reported here.  My 
>> solution to
>> this was to set the $conf[server][sort_limit] variable (IMP 
>> configuration,
>> Server tab) to a small number and basically turn off sorting if there 
>> are
>> too many messages in a  mailbox.  Doing this, resulted in eliminating 
>> the
>> massive request at login.
>
> The bottleneck here is your IMAP server then, not IMP.  RFC 3501 is 
> unfortunately limited in that it does not allow server side sorting 
> for anything other than arrival time.  So to perform sorting on ANY 
> other field requires the IMAP client to download all of the necessary 
> headers from EVERY message in the mailbox to do the sorting on the 
> client-side.  For a disconnected client, this is a potentially brutal 
> operation (it is especially brutal for threaded sort, since it relies 
> on several messages headers at once).  IMP mediates this by being able 
> to cache header data, but you have to enable caching.
>
> This is not a problem with IMP because EVERY client has to do this.  
> You won't see this in desktop clients, however, since they can do 
> things like download message headers in the background and cache these 
> headers.  It is difficult, if not impossible, to do this for a webmail 
> client, however.
>
> Exacerbating the problem is that IMP uses the c-client library to 
> interact with the IMAP server.  Unfortunately, the c-client library is 
> known to be HORRIBLY inefficient when it comes to querying the IMAP 
> server.  To view a single header (say, for example, the "from" header) 
> c-client downloads the entire message envelope of every message in the 
> mailbox, rather than just the header requested.  There is *nothing* we 
> can do about this in IMP 4 since we don't control the IMAP commands 
> being sent to the server.
>
> So now the good news.  First, many modern IMAP servers now support 
> server-side sorting and threading (RFC 5256 - only formally adopted in 
> June 2008).  This eliminates issues related to sorting delays.  Better 
> yet, many of these IMAP servers cache the header information 
> internally so they don't have to parse all mail messages in the 
> mailbox every time a sort request happens.  This drastically speeds up 
> mailbox accesses.
>
> Additionally, c-client is no longer an issue in IMP 5 since we have 
> replaced the c-client library with a native Horde IMAP library that is 
> magnitudes faster.
>
>> What would be nice is if IMP only requested specifics about messages 
>> that
>> appear in the current window rather than ask for every message in the
>> mailbox.  The problem is that if a user selects any sort field other 
>> than by
>> arrival order, then at login, IMP asks for detailed information about 
>> every
>> mail item in the mailbox, slowing down login to a crawl.
>
> This is (unfortunately) correct behavior.  You can't sort a single 
> "message view".  To sort a mailbox requires processing *every* message 
> in the mailbox - or else how would you determine what messages a view 
> entails?  We can work around this a bit - namely, once we get a sorted 
> list for a mailbox, we can continue to use this sort until the mailbox 
> structure changes.  But that only provides partial relief.  The only 
> way to truly remedy this situation is to use an IMAP server that 
> supports RFC 5256.
>
> michael
>


-- 
Jose Manuel Blanco Ramos
European Space Astronomy Centre (ESAC)
European Space Agency (ESA)

E-mail: jmblanco at sciops.esa.int
Tel: +34 91 8131 185, Fax: +34 91 8131 322
P.0. Box 78, 28691 VILLANUEVA DE LA CAÑADA
SPAIN



================================================================================================
This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, 

use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in 

error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by   

the sender. 

Please consider the environment before printing this email.
=================================================================================================



More information about the imp mailing list