[dev] Imap_Client / streams.

[Michael M Slusarz]

Quoting Michael J Rubinsky <mrubinsk at horde.org>:

> I'm having a hard time trying to track down some memory usage issues  
> I'm seeing and in doing so, I'm trying to better understand what is  
> going on in HIC;  When fetching a specific mime part, like e.g. an  
> attachment, does HIC always use streams, or does it put the entire  
> part's contents in memory at some point? E.g:
>
> $bodypart_params = array(
>   'decode' => true,
>   'peek' => true
> );
> $query->bodyPart($mime_id, $bodypart_params);
>
> $fetch_res = $this->_imap->fetch(
>   $this->_mbox,
>   $query,
>   array('ids' => new Horde_Imap_Client_Ids(array($uid)))
> );
>
>
> If $mime_id is some large attachment, will it ever be placed  
> completely in memory or is at contained in a temporary stream at all  
> times?

The Tokenize library was converting literal data from stream ->  
string, before it was stored in a Data_Fetch object.

So first I eliminated the stream -> string conversion in Tokenize by  
copying the literal data to a separate stream.  This is good (max  
memory hit would be 2 MB), but not perfect.

Second step was to not store literal data in the tokenize stream at  
all when reading from the remote IMAP server, and instead store as  
separate stream objects within the Tokenize object.  Now we should  
only be copying once to a stream on reading from IMAP stream to the  
point that the data becomes available in the Data_Fetch object.

This will appear in Horde_Imap_Client 2.28.1.  I'd like to test this  
for a few days before releasing, since if something is broken with  
these changes it will cause all message body data to be improperly  
parsed.  (All unit tests pass, so that is a good sign.)

michael

___________________________________
Michael Slusarz [slusarz at horde.org]