[imp] Only some messages download and display over ActiveSync on phones

[Sebastian Arcus]

On 10/09/2021 00:16, Michael J Rubinsky wrote:
> 
> Quoting Sebastian Arcus <s.arcus at open-t.co.uk>:
> 
>> I am facing a really strange issue with Horde/Imp on mobile devices. A 
>> quick background:
>>
>> The setup is:
>>
>> Horde 5.2.21
>> Lighttpd 1.4.59
>> PHP 7.3.10
>> Dovecot 2.3.7.2
>> Exim 4.92.2
>>
>> This install has been running for years, and as far as I can think 
>> nothing has been updated or reconfigured recently - at least not in 
>> the last few weeks.
>>
>> In the last few days, one of the users has started to experience some 
>> emails not downloading on his mobile devices - for no explainable reason.
>>
>> 1. The emails are showing up fine over IMAP in Thuderbird.
>> 2. The mobile devices with problems are two iPhones, one iPad, and I 
>> just tested it in Bluemail on an Android phone.
>> 3. It isn't always the same messages - if I remove and re-create the 
>> accounts, I get a different random selection of messages downloaded, 
>> after which the sync stops. For example I might have most messages 
>> from today, then yesterday and the day before missing, then some 
>> messages from the day before. There is nothing logical about it.
>>
>> I've checked everywhere I can think of:
>>
>> 1. I've forced resyncing of mailboxes in Dovecot (although I don't 
>> think the problem is there, as all messages show properly in Thunderbird
>> 2. I've changed email passwords
>> 3. I've deleted and re-added email accounts on the iPhone
>> 4. I've hard-reset the iPhones, to try to clear the email cache
>> 5. I've configured the email accounts on an Android phone - still same 
>> issues
>> 6. I've checked Horde ActiveSync error logs - aside from errors about 
>> duplicate sync keys, there doesn't seem to be anything else there.
>> 7. I've tried downloading emails over internal wifi and over 4G
>> 8. I've changed sync span on mobiles between 1 day, 3 days, 1 week, 1 
>> month - the same random behaviour persists
>>
>> This is happening just to this user, who has two email accounts 
>> configured on two iPhones and one iPad. All other 10 users are fine. 
>> Everyone works off the same server, receiving email on iPhones.
>>
>> I've spent 6 hours on this and frankly I can't make any sense of it. 
>> Any suggestions would be much appreciated. Thank you.
> 
> I'd really need to see a sync log and imap debug log to make an educated 
> guess as to what might be happening. Some things that might be related, 
> off the top of my head:
> 
>   * Is this user's prefs set to move deleted messages to Trash or mark 
> them as deleted?
>   * Some memory issue or timeout being hit due a large number/size of 
> messages?
>   * Unlikely, since you say it happens to random messages, but might be 
> some issue with the strategy we use to determine message changes (for 
> Dovecot this would probably be MODSEQ based).
> 
> If you can get me a copy of the sync log for this user I might be able 
> to help narrow down the issue. Feel free to mail me directly 
> mrubinsk at horde.org.

Just checking if you got my email with the logs. The phone is still 
syncing the emails completely erratically. The only error I keep on 
seeing in the logs is:

2021-09-16T13:36:21+01:00 ERR: HORDE SQL QUERY FAILED: SQLSTATE[23505]: 
Unique violation: 7 ERROR:  duplicate key value violates unique 
constraint "horde_activesync_state_pkey"

Or any other suggestions I could try? Thank you in advance