[Tickets #13747] Re: Speed up ActiveSync synchronisation handling

[noreply at bugs.horde.org]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: https://bugs.horde.org/ticket/13747
------------------------------------------------------------------------------
  Ticket             | 13747
  Updated By         | Michael Rubinsky <mrubinsk at horde.org>
  Summary            | Speed up ActiveSync synchronisation handling
  Queue              | Synchronization
  Version            | Git master
  Type               | Enhancement
-State              | New
+State              | Feedback
-Priority           | 2. Medium
+Priority           | 1. Low
  Milestone          |
  Patch              |
  Owners             |
------------------------------------------------------------------------------


Michael Rubinsky <mrubinsk at horde.org> (2014-12-09 18:04) wrote:

> Dear Horde team,
> I have noticed some performance issues in Horde with email "power users"..
>
> "Power users" are users having > than 150 message folders and 25k  
> messages in them.. (just to have an idea about what I'm speaking)
>
> Every time a power user logs in via ActiveSync (Outlook 2013, ..) the

I would fall into this category, though not ALL of my folders have  
that many messages. I don't see such a slowdown, but I do not use  
Apache.

> apache service rises up to 100% CPU usage for 3-6 seconds.
> Also, every time a power user mark just 1 message as read/unread the  
> same problem arise.

> The bottleneck seems to be just the Horde service (apache).
> There is practically no access to harddisks and/or other hardware  
> during the CPU usage spike.

FIrst, I am going to assume you are using an efficient  IMAP server  
such as Dovecot that supports, among other things, modification  
sequences. Otherwise, your performance for such a large mailbox will  
degrade.

The way the mail synchronization works for such an IMAP server (and  
I'm heavily simplifying) is that we remember the last MODSEQ of each  
mailbox and then basically perform a time-escaped loop that queries  
the IMAP server for changes in each folder since that folder's MODSEQ.  
  After each folder is queried (assuming no changes were found) we  
sleep a configurable number of seconds and then repeat the process.  
The loop (and eventually the request) exit either when a change is  
found, or the heartbeat interval is reached. There is no "looping over  
every message in the folder" as you had asked in the mailing list.  
Checking each folder for changes (should be) a very fast operation -  
limited by your IMAP server's performance. For large number of  
folders, each loop iteration might take some time, but there is  
nothing we can about that since it is the only way to detect changes.  
A unique-ish thing about OL is that it ALWAYS polls ALL folders. It's  
not configurable like just about every other EAS client out there.

On the Horde side, the only thing I can think of for such a large  
number of messages is that it might be related to the state storage  
since even for MODSEQ servers, we need to cache a (compressed) list of  
message uids that are on the device.

> Is there any possibility to investigate further?

You should investigate using a tool such as xdebug to profile where  
this time is being spent.

> Could there be some needless loops in Horde that could be optimized?

Anything is possible. Again, xdebug will be your friend here.