[Tickets #3359] Re: Make use of streams for speed and memory efficiency

[bugs at horde.org]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/3359
------------------------------------------------------------------------------
  Ticket             | 3359
  Updated By         | Michael Slusarz <slusarz at horde.org>
  Summary            | Make use of streams for speed and memory efficiency
  Queue              | Horde Framework Packages
  Version            | HEAD
  Type               | Enhancement
  State              | Assigned
  Priority           | 2. Medium
  Milestone          | Horde 4.0
  Patch              |
  Owners             | Michael Slusarz
------------------------------------------------------------------------------


Michael Slusarz <slusarz at horde.org> (2009-03-26 01:43) wrote:

Some notes on some testing:

Will want to use this format:
php://filter/read=convert.base64-encode/resource=file:///tmp/foo

See http://us.php.net/manual/en/wrappers.php.php

Don't use file_get_contents()/stream_get_contents().  On my test file  
(11 MB of text data), file_get_contents() required 23 MB.  This is  
much more efficient:

while (!feof($a)) {
     $b .= fread($a, 8192);
}

This used only 300-400KB over the file size.

Same with writing - don't use file_write_contents() for large data.   
fwrite in chunks (4096 is probably good).

And use temp stream.  temp will copy to memory for first X MB  
(configurable) and then will write to disk.  Reading the 11 MB file  
via the method above, and then writing to temp stream using fwrite(),  
and then rewinding temp stream pointer and reading entire file using  
fread() took only 14 MB total.