[dev] Horde IMAP features (or why our webmail client is better than your webmail client)

[Michael M Slusarz]

Been meaning to do this for awhile, and this is only sort of a  
half-hearted first attempt/draft, but in the past there has been  
interest into what has actually gone on internally in Horde 4/IMP 5 to  
improve performance and features.  Answer: a bunch.

I can proudly state that Horde_Imap_Client, the Horde library that  
powers IMP, is by far the most complete, fastest, and most feature  
rich PHP library out there.  As a webmail developer, I often monitor  
the other freely available options there to gauge our process.  And my  
honest assessment is that we are unarguably the best solution when it  
comes to features, performance, and stability.

So it might be good to list what I think some of these features are so  
that others can compare and see for themselves what we do better than  
anyone else.  To further this goal, I've started to add some content  
to the Horde_Imap_Client page:

http://wiki.horde.org/Project/HordeImapLib

If there is interest, I will try to expand on some of the existing  
entries (and add additional information).  I've thought about  
providing side-by-side outputs of things like IMAP transaction logs  
with other clients (webmail or desktop) to show our efficiency, but  
I'm wondering if that might be too esoteric for most people - IMAP is  
an incredibly complex protocol so the benefits may not be immediately  
or easily recognizable.  But if there is enough interest, I would  
consider providing these kind of demonstrations.

For those that don't care about any of the above, and want a distilled  
version, I'll present the three things I am most proud about the  
Horde_Imap_Client library:

1. PHP Memory stream support.  Unlike every other PHP library I am  
aware of, our library does not keep the full data of body parts in  
memory.  Since attachments can be huge (10-20 MB), this could cause  
memory issues on the server.  Those running IMP 4 might be aware of  
this - memory limits for PHP had to be set high, partially because the  
PHP imap extension commands we used required all data to live in  
memory.  However, Horde_Imap_Client instead stores body part data in  
temporary streams.  These streams only store a limited amount of data  
in memory (2 MB), with the rest being spooled to a temporary file  
somewhere on your local system.  This results in vastly reduced memory  
loads when viewing/downloading message body parts.  As an example - I  
have had my memory limit set to 64 MB for over a year now.  I have not  
once run out of memory (as a developer, I often play around with large  
messages just to test for things like these).

2. CONDSTORE/QRESYNC support.  These are two recent IMAP extensions  
specifically created to add disconnected clients (like IMP) keep their  
local caches synced and to speed viewing when reloading a mailbox.   
IMP is the *ONLY* open-source webmail client that I am aware of that  
supports these extensions.  Discussing the exact details is beyond the  
scope of this message, but absence of CONDSTORE/QRESYNC support means  
two things:
   * You can not properly cache mailbox data.  Namely, flags are not  
guaranteed to be synced unless they are downloaded EVERY time you  
access a mailbox.  Which is very slow.  So, in other words, all other  
webmail clients are essentially "broken" - only IMP does caching  
correctly.
   * You can not properly keep an AJAX display synchronized.  A  
well-written AJAX client contains yet another cache of mailbox data -  
it needs to be synchronized with the PHP server (which in turn needs  
to be synchronized with the IMAP server.  Confused yet?)  Without  
CONDSTORE/QRESYNC, this can not be done correctly absent the AJAX  
interface *constantly* reloading data from the server.  But this is  
just a waste of bandwidth, processing power, and the user's time.   
Only IMP does AJAX display correctly because we only download what is  
needed.

3. Fast. The PHP imap (c-client) extension is notoriously bad at its  
IMAP conversations.  It would send duplicate requests, couldn't cache  
entries, and would request data for messages that you didn't even care  
about (the infamous lookahead "feature").  Additionally, there were  
many things that couldn't be done at all - like obtaining namespace  
information on the server - because the extension did not support it  
so an entirely separate library needed to be written, and and an  
entirely separate connection to the IMAP server needed to be made, to  
fill in the gaps.  Horde_Imap_Client has replaced all this was a  
single, coherent library that has been relentlessly tuned to ensure  
that we are doing things the most efficient way possible.

This is probably a good place to end this message.  I try not to be  
braggy very often about the projects I am involved in, but now that  
IMP 5 has begun to stabilize I figure the next step is to try to  
inform people why it should be their choice.

michael

___________________________________
Michael Slusarz [slusarz at horde.org]