[imp] Only some messages download and display over ActiveSync on phones

[Sebastian Arcus]

On 10/09/2021 00:16, Michael J Rubinsky wrote:
> 
> Quoting Sebastian Arcus <s.arcus at open-t.co.uk>:
> 
>> I am facing a really strange issue with Horde/Imp on mobile devices. A 
>> quick background:
>>
>> The setup is:
>>
>> Horde 5.2.21
>> Lighttpd 1.4.59
>> PHP 7.3.10
>> Dovecot 2.3.7.2
>> Exim 4.92.2
>>
>> This install has been running for years, and as far as I can think 
>> nothing has been updated or reconfigured recently - at least not in 
>> the last few weeks.
>>
>> In the last few days, one of the users has started to experience some 
>> emails not downloading on his mobile devices - for no explainable reason.
>>
>> 1. The emails are showing up fine over IMAP in Thuderbird.
>> 2. The mobile devices with problems are two iPhones, one iPad, and I 
>> just tested it in Bluemail on an Android phone.
>> 3. It isn't always the same messages - if I remove and re-create the 
>> accounts, I get a different random selection of messages downloaded, 
>> after which the sync stops. For example I might have most messages 
>> from today, then yesterday and the day before missing, then some 
>> messages from the day before. There is nothing logical about it.
>>
>> I've checked everywhere I can think of:
>>
>> 1. I've forced resyncing of mailboxes in Dovecot (although I don't 
>> think the problem is there, as all messages show properly in Thunderbird
>> 2. I've changed email passwords
>> 3. I've deleted and re-added email accounts on the iPhone
>> 4. I've hard-reset the iPhones, to try to clear the email cache
>> 5. I've configured the email accounts on an Android phone - still same 
>> issues
>> 6. I've checked Horde ActiveSync error logs - aside from errors about 
>> duplicate sync keys, there doesn't seem to be anything else there.
>> 7. I've tried downloading emails over internal wifi and over 4G
>> 8. I've changed sync span on mobiles between 1 day, 3 days, 1 week, 1 
>> month - the same random behaviour persists
>>
>> This is happening just to this user, who has two email accounts 
>> configured on two iPhones and one iPad. All other 10 users are fine. 
>> Everyone works off the same server, receiving email on iPhones.
>>
>> I've spent 6 hours on this and frankly I can't make any sense of it. 
>> Any suggestions would be much appreciated. Thank you.
> 
> I'd really need to see a sync log and imap debug log to make an educated 
> guess as to what might be happening. Some things that might be related, 
> off the top of my head:
> 
>   * Is this user's prefs set to move deleted messages to Trash or mark 
> them as deleted?
>   * Some memory issue or timeout being hit due a large number/size of 
> messages?
>   * Unlikely, since you say it happens to random messages, but might be 
> some issue with the strategy we use to determine message changes (for 
> Dovecot this would probably be MODSEQ based).
> 
> If you can get me a copy of the sync log for this user I might be able 
> to help narrow down the issue. Feel free to mail me directly 
> mrubinsk at horde.org.

A bit of an update on my own post. Looking through the logs, it seems 
that the two iPhones and the Andoid phone (which I've setup for 
testing), they all use the same SyncKey for each email account. Could 
that be the possible culprit? Could there be some race condition where 
the server gets confused as to which device has synced/seen which messages?