[imp] Cannot download large folder

[Otto Stolz]

Hello,

Andrew Morgan schrieb:
> I'm almost positive that the memory usage doubles temporarily for this 
> function call:
>           $body = str_replace("\r\n", "\n", $body);
> 
> However, I was able to avoid this memory usage penalty by calling 
> str_replace within the loop on each message, rather than the entire huge 
> string at the end.  This also uses less memory overall.

I have modified my Imp H3 (4.1.3) test installation (under PHP 5.2.0
and openSUSE 10.2 (i586)) in the way outlined by Andrew. Then I have
logged both the MBox size and the Memory Peak Size, as the Mbox was
assembled.

With one particular folder (size: 37.6 MB), I found
   M ≤ 2.1 × b + 8.6 MB
where:
   M : Memory peak size
   b : Size of Mbox assembled so far

Interestingly, there are plateaus in the curve, where generateMbox
will not claim more memory as the Mbox is growing, but then the
Memory usage will jump up to the original straight line, cf. attached
diagram.

With two smaller mboxes, I have found:
   M ≤ 3,0 × b + 8.3 MB  ( b ≤ 15.4 MB)
   M ≤ 3.0 × b + 8.6 MB  ( b ≤ 14.3 MB)
These are also plotted into the attached diagram.

Hence I guess, 4 × b + 10 MB will suffice with the modified algorithm,
in any case. However, the test with my customers monster folder (81 MB)
is still to come.

Best wishes,
   Otto Stolz


-------------- next part --------------
A non-text attachment was scrubbed...
Name: generateMbox.png
Type: image/png
Size: 7547 bytes
Desc: not available
Url : http://lists.horde.org/archives/imp/attachments/20070720/32aeacf6/attachment.png