[dev] STATUS_SYNC* discussion (Was: [Tickets #11612] Re: Broken imap fetch query)

[Michael J Rubinsky]

Moving to dev@ since this is not related to the bug that was fixed.

> Michael Slusarz <slusarz at horde.org> (2012-10-31 17:07) wrote:
>
>>> FYI: depending on what you are doing with this return, it may be much
>>> faster to use STATUS_SYNCFLAGUIDS and/or STATUS_SYNCVANISHED instead
>>> - if using Imap_Client's cache, this methods eliminate 1-2 calls to
>>> the remote server.
>>
>> Thanks for the tip. At the point in the code where this is (was)  
>> happening, I need to know about new/changed messages as well as  
>> vanished messages. For the changed messages I need the actual state  
>> of any flags so I don't think I can use the STAUS_SYNCFLAGUIDS. I  
>> might be able to make use of the STATUS_SYNCVANISHED value, as well  
>> as tweak other parts of the code where I am only "pinging" the  
>> server to check for the existence of a change without needed to  
>> know what the change is.
>
> IIRC, you are not using Imap_Client's built-in caching?  If no, then  
> these don't do anything for you (since QRESYNC/CONDSTORE isn't  
> syncing on mailbox open).

I do some switching based on the availability of QRESYNC and use the  
STATUS_HIGHESTMODSEQ value when it's available. At least at the time I  
wrote the code, QRESYNC was only being reported by Horde_Imap_Client  
when caching was turned on, so in a way I'm relying on the cache being  
available, but not actually using the data. I track these myself since  
I need to persist the values across any number of different sessions.  
In other words, I need to know values such as HIGHESTMODSEQ and/or  
UIDNEXT at the time that ActiveSync last checked, not since the  
mailbox was first opened during the current request. In some cases  
like e.g., during PING requests, I'm only interested if some change  
has occurred, not what the exact change is. For that, I only take  
UIDNEXT into consideration for performance reasons. Only after some  
change has been detected does ActiveSync request the change specifics  
- at which point I want to know all the changes available, including  
VANISHED and flag changes.


I will need to look at this more closely, since it sounds like I might  
be able to write some logic to first synchronize/compare *ActiveSync*  
code's last known MODSEQ with the *IMAP* client's STATUS_SYNCMODSEQ on  
the first access of each request and then for every other check during  
the PING loop simply check STATUS_SYNCMODSEQ. To further complicate  
things, I need to track different versions of the cached  
MODSEQ/UIDNEXT values. ActiveSync uses a sync_key to specify the last  
known state of the collection being synched and we need to keep at  
least two of these keys cached since the device can request the old  
sync_key if it never received the data in full or there was some other  
communication issue - not uncommon on a mobile device.

> But see Ticket #11590 - If not using Imap_Client's full caching,  
> there is still an opportunity to take advantage of this.  Namely:  
> creating a custom cache driver that has the known MODSEQ value that  
> can be returned from it.  The mailbox will be sync'd on open, and  
> the flag values can be cached in the custom cache driver so that  
> SYNCFLAGUIDS/VANISHED doesn't need to hit the server again (the  
> custom cache driver could be an in-memory cache).

Hmm, I could use/refactor ActiveSync's current caching object  
(Horde_ActiveSync_Folder_Imap) as a custom cache driver and let  
activesync decide which sync_key's data to populate it with before  
injecting it into the imap client. I know this will let me capture  
changes after the mailbox syncs, but how would I capture the changes  
during that first sync, by a normal fetch query using the MODSEQ that  
I have?

I'm probably misunderstanding something but for example:
ActiveSync has a last known MODSEQ of 100 and hasn't connected in  
e.g., a week. During that time the mailbox's MODSEQ has changed to  
200. During that first connection, STATUS_SYNCMODSEQ will be set to  
200 so any of the STATUS_SYNC* values will be compared to 200, not to  
100, right? How do I get that first change set without having to write  
switching code to determine if we are using a cache etc...?

> I'm probably going to change the internals of Imap_Client soon so  
> that all objects cache - if not given a cache parameter, than FETCH  
> items are only cached for that page access.  However, this won't  
> take advantage of the fact that you have a MODSEQ value/saved state  
> before even connecting to the mailbox.


-- 
mike

The Horde Project (www.horde.org)
mrubinsk at horde.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6062 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.horde.org/archives/dev/attachments/20121031/6860296e/attachment.bin>