[Tickets #12506] Re: Contacts sync with ActiveSync get duplicates and not sync

[noreply at bugs.horde.org]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/12506
------------------------------------------------------------------------------
  Ticket             | 12506
  Updated By         | Michael Rubinsky <mrubinsk at horde.org>
  Summary            | Contacts sync with ActiveSync get duplicates and not
                     | sync
  Queue              | Synchronization
  Version            | Git master
  Type               | Bug
  State              | Resolved
  Priority           | 1. Low
  Milestone          |
  Patch              |
  Owners             | Michael Rubinsky
------------------------------------------------------------------------------


Michael Rubinsky <mrubinsk at horde.org> (2013-08-04 17:00) wrote:

> The issue is that Horde "buffers" the replies and send them all at a  
> time later. I have always assumed this is right because you need the  
> status for the whole Sync and you can not know this until all  
> additions are processed.

This is partially correct. We also need to ensure that the entire  
request is handled before we send any responses and the ADD command  
may not be the only or last command processed. We process the incoming  
data as we receive it so that we don't need to keep all the incoming  
message data in memory before we start processing it.

> By examplle If I use the uploaded patch (hack) it only process the  
> first 50 ADDs in the packet, this sortens the GAP to less than 1  
> minute and workarounds the timeout. With this patch (hack) Android  
> seems to work fine and sync all contacts with time.

This patch would completely break synchronization on any EAS client  
that actually follows the specs. It ignores incoming changes and sends  
no status response on those skipped messages. According to the specs  
the client MUST assume the addition worked and should NOT send the  
change(s) again.

>> What probably should happen is when a duplicate addition is detected
>> we should return the existing item's UID as if the addition was
>> successful. Of course, I have no idea what your client will do with
>> this, since it is demonstrating broken behavior anyway, not to
>> mention if the connection continues to timeout, this won't help your
>> original problem anyway.

> This sounds really interesting, I would test it, I think it should  
> work, there are not additions so the GAP should be low.

Thinking about this some more, I don't think this is the answer  
either. When these sync with the server if we send back the same UID  
for each one of these (an action which is undefined in the EAS spec),  
there could now be multiple copies in the client's datastore that  
connect with the same UID on the backend. What happens when a user  
deletes one of these? Yup, the (only) copy of it on the server is  
deleted. Less worrisome is that there would still be copies of it on  
the client not attached to any server side copy. The state is now  
corrupt. Even if the client doesn't exhibit the broken behavior yours  
does of sending different clientids, it would still be possible for  
the user to add the same contact data accidentally.

If anything is to be done here to mitigate this, I think we need to  
totally rewrite the way incoming changes are handled so that we don't  
actually process them until they are all read. We will use more memory  
on the incoming request but we will be able to start sending packets  
back to the client more quickly.  I'm not sure I like this, as one of  
the things that differentiates Horde's activesync implementation is  
relatively low memory consumption per request and I have not been able  
to find any protocol related documentation as to any incoming change  
time limits. I think the use case of importing hundreds and hundreds  
of new contacts FROM THE CLIENT is relatively rare.