[imp] Excessive memory usage

[Andrew Morgan]

On Wed, 24 Sep 2008, Michael M Slusarz wrote:

> It's a known issue in that our current MIME handling regime isn't the most 
> efficient in handling what I like to call "embedded" MIME data.  "Embedded" 
> data is defined as any MIME part that contains data that can't be viewed 
> until it itself is translated/decoded somehow (this is in addition to the 
> original MIME decoding).  Additionally, the c-client library is known to be 
> memory hungry when handling message part data so this adds to the problem.

Yeah, I can see why that would be a problem.  :)

> As for the c-client issue, this problem is presently being worked on (a fully 
> PHP stream-based IMAP client has nearly reached beta stage and will be used 
> in the forthcoming IMP 5).  The MIME handling is also slated to be improved. 
> But the major issue in your case is that nobody (and I mean nobody) should be 
> sending attachments using appledouble.  There is nothing gained by using 
> appledouble that can't be handled by a MIME multipart/mixed message.  I 
> realize that you can't control what mail programs people are using, but at 
> some point you *do* have to say "I'm sorry, we just can't handle that 
> message."

Haha yeah this message is evil is many ways...  Within our system, I limit 
the messages to 10MB during compose, but outside folks send bigger stuff 
still (100MB limit on our relays).

> And before you think this is an IMP specific issue, I have several test 
> e-mail messages that are 40 MB or so that when I view with Thunderbird (on my 
> Quad Core Q6600 w/ 3 GB of RAM), it absolutely brings my machine to its 
> knees.  This is an issue with the mail transport architecture in general, not 
> just Horde/IMP.

Nah, I'm a huge fan of IMP.  But unlike other email clients, I can 
actually modify the code easily and contribute improvements.  :)

I'll take a look at the appledouble code and see if there is any way I can 
reduce the memory requirements.

 	Andy