[horde] cpu usage 80-85 in imp Urgent

[Jeff Warnica]

Again, with the right From address....

On Sun, 2005-02-01 at 22:34 +0530, Joel n.solanki wrote:

> 'name' => 'IMAP Server',
>     'server' => 'localhost',
>     'protocol' => 'imap/notls',
>     'port' => 143,
>     'folders' => 'mail/',
>     'namespace' => '',
>     'maildomain' => 'my.net',
>     'smtphost' => 'mail.my.net',
>     'realm' => '',
>     'preferred' => ''
> 
> I have already set this to mail/ 
> my mails gets dropped at /home/$user/Mailbox
> 
> all other folders are in mail/ according to imp horde.

Something in there just doesn't sound right to me. But you haven't said
the users cant get their mail, or that users files show up in IMP, so
its probably correct.

> my users have about 250 to 300 mb mails in inbox. 

That is a large amount of mail...

> 
> i think when imp loads it takes all mails in memory so there it is using
> this much cpu ..

Well, thats not happening. For one, PHP would, by default, die with a
memory limit error if it got even to a small fraction of that... And its
just not what IMP does.

I haven't used UW-IMAP for a long time, but speculating from experience:
The load is because IMP reloads the headers, and other meta information
basically on every index page load. Outlook, and most other client
client, would cache that type of information locally, even if they don't
store the messages locally. So every time IMP has to display a folder
index page, it has to have UW-IMAP scan through an entire folder, and on
the portal page and in the menu, through every folder. It is also
possible, and this is just a wild guess, that because this takes so
long, either PHP itself is timing out, or the user hits cancel, and then
the search is restarted.

Are you using "traditional Unix mailboxes"? That is, one file per
folder, plan text? If so, I hate to break it to you, but that is about
the most stupid format available. Its the default, it works, and its
simple, but its slow, prone to breaking, and has lots of other
problems... Ill put it to you this way: in 1990 when UW-IMAP was first
produced the author intentionally did not support what then was the
predominate format, the "traditional" Unix mailbox format because of all
its problems.

>  
> 
> any solution ..suggestion


As I said, that is a lot of mail per user. I would investigate some
better mail storage system. Personally, I would recommend Cyrus IMAP.
That can be somewhat complex to deal with, you probably have all the
software you need already to use, say, maildir, or some other format.
Migration would be painful.

But that doesn't help you today, or tomorrow. One thing that I would try
would be to see if an IMAP proxy would help at all. My theory is that if
there are long searches, with PHP or users aborting them, if IMAP proxy
keeps the connection open, then UW-IMAP either will not abort the
search, or at least keep the aborted search results in memory. This also
has the advantage of being transparent, relatively easy, and easily
undoable.

http://www.horde.org/imapproxy/