[Tickets #2341] NEW: Performance issue during recursive folder traversal

[bugs@bugs.horde.org]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/?id=2341
-----------------------------------------------------------------------
 Ticket             | 2341
 Created By         | maurer at amasol.de
 Summary            | Performance issue during recursive folder traversal
 Queue              | IMP
 Version            | 4.0.3
 State              | Unconfirmed
 Priority           | 2. Medium
 Type               | Bug
 Owners             | 
-----------------------------------------------------------------------


maurer at amasol.de (2005-07-26 08:55) wrote:

After upgrading to IMP 4.0.3 (from IMP 3.2.1), I noticed a severe
performance degradation.  After some analysis, I noticed that IMP send large
amounts of LSUB queries to our IMAP server.  Most of these are unnecessary.

This is a particular big problem with our support account.  This account has
about 2000 folders that are placed in a 2-3 level folder hierarchy with
about 80 non-leaf nodes.

IMP starts with listing all subscribed folders using
  00000003 LSUB "" *
Then, however, it starts to query every single folder, including the leaf
folders:
  00000007 LSUB "" some/folder/%
I.e. that results in about 2000 LSUB requests that put a high load on both
the IMAP server and the Apache/PHP installation.

Since the imap-uw server that I was using didn't support the HasNoChildren
flag in the LSUB response, I decided try courier-imap.  However, the number
of LSUB requests doesn't change:
  00000007 LSUB "" INBOX.some.folder.%
This server correctly returns the HasNoChildren flag on leaf nodes
  * LSUB (\HasNoChildren) "." "INBOX.some.folder"
but IMP apparently doesn't utilize this information (in this particular
piece of code).

I'd suppose that the problem is somewhere in the recursive folder tree
building code, probably somewhere around _getList - but that's just my
guess.

IMP 3.2.1, OTOH, only used
  00000003 LSUB "" *
to get the complete list of folder and deducted everything else from that
information.

I'd like to suggest that IMP either properly supports the HasNoChildren flag
or - preferably - caches the results of the *-query and dispenses with
subsequent calls of LSUB.