[horde] Problem of slow with server-side sorting

[Andrew Morgan]

On Tue, 2 Jun 2009, Vilius Šumskas wrote:

>> I thank you all for your reply.
>>
>>  Could configure "Arrival Time"
>> in the accounts, but I like to go deeper into the solution of this
>> problem, know the real cause of the slow, perhaps because the server
>> hardware, the courier-imap, etc..
>>
>>  We would appreciate if you could help me because the problems are
>> becoming massive.
>
> I *think* the root cause is Courier IMAP, because as I told you earlier it
> doesn't handle large mailboxes well. Why is it so I'm not sure. Maybe it is
> related to internal Courier caching during expensive operations. All I know
> that slowness was the main reason why we moved to Cyrus.

Courier is slow on large mailboxes because it uses the "maildir" format 
for storing email.  In this format, each message is a separate file.  When 
you ask the server to sort by something other than arrival time, it must 
open each message file to read the message headers.  As you can imagine, 
it takes a long time to open several thousand files and read their 
contents.

> I just did quick test on a Cyrus mailbox with 7700+ messages on the machine
> with Operator processor, 2GB of RAM and SATA RAID5 array. It took less than
> 3 seconds.

Cyrus improves on maildir by also creating additional files in each 
directory named cyrus.index and cyrus.cache.  These files contain the 
message headers so that Cyrus can quickly retrieve the headers from a 
single file.

 	Andy