[commits] [Wiki] changed: ActiveSync/Development
Michael Rubinsky
mrubinsk at horde.org
Thu Nov 24 00:44:05 UTC 2016
mrubinsk Thu, 24 Nov 2016 00:44:05 +0000
Modified page: https://wiki.horde.org/ActiveSync/Development
New Revision: 11
Change log: Basic flow of fetching changes.
@@ -15,29 +15,29 @@
The following is a general overview of the important classes in this
library and what they are responsible for:
: Horde_ActiveSync : This is the main "server" and the entry point
for the library. Also contains some getters for various objects used
throughout.
-: Horde_ActiveSync_Collections : This class is essentially a manager
for "collections". A collection is what we call a group of properties
related to a single synchable data store (such as an email folder, or
a contact list). It is the entry point for the SYNC and PING handlers
for asking for changes. It manages the SyncCache, is responsible for
ensuring we have all needed information when handling empty SYNC
requests, detects concurrent requests, and a bunch more.
+: Horde_ActiveSync_Collections : This class is essentially a manager
for "collections". A collection is what we call a group of properties
related to a single synchable data store (such as an email folder, or
a contact list). It is the entry point for the SYNC and PING handlers
for asking for changes. It manages the !SyncCache, is responsible for
ensuring we have all needed information when handling empty SYNC
requests, detects concurrent requests, and a bunch more.
: Horde_ActiveSync_Device : Manages device information such as OS
and version. Also contains code that works around certain quirks that
are specific to certain clients.
: Horde_ActiveSync_Mime : A wrapper around Horde_Mime_Part that adds
to and overrides existing behavior to make it more specific to
ActiveSync. There is also Horde_ActiveSync_Mime_Iterator that is
changed from the the normal Horde_Mime_Iterator behavior in that we
don't include what EAS considers attachments to be returned, and we
also include the base part when iterating (whereas Horde_Mime does
not). Objects are instantiated from Horde_ActiveSync_Imap_* classes.
: Horde_ActiveSync_Rfc822 : Deals with handling RFC822 email message
strings/streams in the context of EAS. E.g., used when handling
incoming SENDMAIL commands.
-: Horde_ActiveSync_SyncCache : This class is responsible for
storing/loading the SyncCache data from the storage/state driver. The
SyncCache started life as a way of caching data from SYNC and PING
requests that would be needed if the client issued an "empty"
SYNC/PING request. Empty requests are required to use the same values
as the last non-empty request. It has since grown to be essentially a
shared data store that helps keep collection data up to date between
any and all running requests. Most (if not all) interaction with this
class is through the Horde_ActiveSync_Collections object.
+: Horde_ActiveSync_SyncCache : This class is responsible for
storing/loading the !SyncCache data from the storage/state driver. The
!SyncCache started life as a way of caching data from SYNC and PING
requests that would be needed if the client issued an "empty"
SYNC/PING request. Empty requests are required to use the same values
as the last non-empty request. It has since grown to be essentially a
shared data store that helps keep collection data up to date between
any and all running requests. Most (if not all) interaction with this
class is through the Horde_ActiveSync_Collections object.
: Horde_ActiveSync_Connector : These classes are responsible for
sending/receiving the structured WBXML for certain requests. E.g., the
H_AS_C_Exporter_Sync object contain methods that are called from the
H_AS_Request_Sync object that send the WBXML needed for sending each
change. It is passed the message object, encodes it to WBXML, and
wraps it with any needed response WBXML before sending it down the
output stream. Likewise, the H_AS_C_Importer object reads the
structured WBXML for importing changes. These methods are called from
the Horde_ActiveSync_Request_* classes after the changes have been
fetched.
: Horde_ActiveSync_Driver_Base : This is the base class that
Horde_Core_ActiveSync_Driver extends.
: Horde_ActiveSync_Folder_* : These classes represent a specific
collection's state. It's an abstraction around the state data and
represents the point in time of the sync represented by the synckey.
For non-email collections, this basically stores the last known
syncstamp/time along with a few other details. For email folders, this
class stores things like the HIGHESTMODSEQ, UIDVALIDITY, NEXTUID, and
the list of IMAP UIDS that the are on the client etc... If not using a
server that supports CONDSTORE then this will also contain flag
information. Also contains logic used to set/get/calculate what UIDs
have changed/vanished etc... These objects are used to transport the
change set back and forth from the backend driver to the ActiveSync
code that fetches and sends each detected change.
-: Horde_ActiveSync_Imap_* : These classes interact with the IMAP
server. H_AS_I_Adapter contains the bulk of the logic for fetching
changes, messages, etc.... The Message object wraps a single IMAP
message and the MessageBodyData object abstracts access to the
message's body taking various things into account (if the client needs
HTML body, if it needs truncation) and ensuring the data is proper
UTF-8 data.
+: Horde_ActiveSync_Imap_* : These classes interact with the IMAP
server. H_AS_I_Adapter contains the bulk of the logic for fetching
changes, messages, etc.... The Message object wraps a single IMAP
message and the !MessageBodyData object abstracts access to the
message's body taking various things into account (if the client needs
HTML body, if it needs truncation) and ensuring the data is proper
UTF-8 data.
: Horde_ActiveSync_Message_* : These classes represent either an
actual message item i.e., a Calendar item or a part of a message item
that is contained by a message item i.e., and Attendee or a Flag. Each
class is responsible for knowing how to encode itself into WBXML or to
decode a WBXML stream into the item's properties.
-: Horde_ActiveSync_State_* : This is the state/storage driver.
Responsible for interacting with the persistent storage used by
ActiveSync (normally Sql). This is another one of those classes that
has grown too big for it's original name and needs to be broken down
for H6. Started as a driver for maintaining the state/synckey
"sync-points" but has grown to encompass managing all storage needs.
E.g., the SyncCache is actually persisted to storage using this class.
+: Horde_ActiveSync_State_* : This is the state/storage driver.
Responsible for interacting with the persistent storage used by
ActiveSync (normally Sql). This is another one of those classes that
has grown too big for it's original name and needs to be broken down
for H6. Started as a driver for maintaining the state/synckey
"sync-points" but has grown to encompass managing all storage needs.
E.g., the !SyncCache is actually persisted to storage using this class.
+++ Horde_Core
Core contains any code specific to handling Horde Groupware
collections. This is where requests for information and changes to
information are actually handled.
@@ -137,10 +137,12 @@
++ Life Cycle of a Client.
What we will describe is the life cycle of a client-server pairing
from the initial connection to be able to synchronize changes. Let's
start with a fresh, never before connected client. The details and
order of things vary from EAS version to version. This is designed as
a general overview and most closely resembles what happens in EAS
version >= 14.
++++ OPTIONS
The first thing that happens is the OPTIONS request. This
essentially tells the server what protocol versions the client
supports and the server responds with (among other things) the version
that it will be using. This is usually where the device object is
first created. TODO: Flesh this out with more details.
++++ SETTINGS
Second will normally be a SETTINGS request. This is where the client
can give the server details about the device and/or OS, application
name etc... Normally this is responded to with server details about
the user account. The following is an example synclog of the SETTINGS
request and response using a Windows 10 Mail client:
<code>
2016-11-14T17:03:59+00:00 INFO: [10030]
Horde_Core_ActiveSync_Driver::authenticate() attempt for mike
@@ -220,9 +222,9 @@
2016-11-14T17:03:59+00:00 DEBUG: [10030] O </Settings:UserInformation>
2016-11-14T17:03:59+00:00 DEBUG: [10030] O </Settings:Settings>
</code>
-
++++ FOLDERSYNC
For the purposes of this page, we are going to ignore the
PROVISIONING stuff. The next step is the FOLDERSYNC request:
<code>
2016-11-14T17:03:59+00:00 DEBUG: [10029] I <FolderHierarchy:FolderSync>
2016-11-14T17:03:59+00:00 DEBUG: [10029] I <FolderHierarchy:SyncKey>
@@ -230,9 +232,9 @@
2016-11-14T17:03:59+00:00 DEBUG: [10029] I </FolderHierarchy:SyncKey>
2016-11-14T17:03:59+00:00 DEBUG: [10029] I </FolderHierarchy:FolderSync>
</code>
-First, the client requests a SyncKey of 0. This indicates it is a
fresh start and that the server should discard any previous sets of
state it may have for this client and user. The server fetches the
list of folders that are available. For this purpose each groupware
share (address book, calendar etc...) is considered a folder. The
entry point for this is
//**Horde_Core_ActiveSync_Driver::getFolderList()**//.
+First, the client requests a !SyncKey of 0. This indicates it is a
fresh start and that the server should discard any previous sets of
state it may have for this client and user. The server fetches the
list of folders that are available. For this purpose each groupware
share (address book, calendar etc...) is considered a folder. The
entry point for this is
//**Horde_Core_ActiveSync_Driver::getFolderList()**//.
At this point we should talk about the backend's id and the
ActiveSync UID for each folder. From the client's point of view, all
it cares about is the uid for the folder. We create new UIDs whenever
a new folder is encountered. The mapping of UIDs to backend ids is
managed by the Horde_ActiveSync_State driver. So, you will see
something like this in the sync log:
<code>
@@ -277,13 +279,14 @@
2016-11-14T12:04:01-05:00 DEBUG: [10029] O </FolderHierarchy:Changes>
2016-11-14T12:04:01-05:00 DEBUG: [10029] O </FolderHierarchy:FolderSync>
</code>
-This is a good time to mention the SyncKey. The SyncKey is the unique
identifier a specific set of state. It represents the state of the
collection being synced at that point in time. It always has the
following form:
+This is a good time to mention the !SyncKey. The !SyncKey is the
unique identifier a specific set of state. It represents the state of
the collection being synced at that point in time. It always has the
following form:
{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}y where y is a continuously
increasing counter. That increments any time changes are received from
or sent to the client. The GUID-looking value in front of the counter
will remain constant after it's generated for each collection.
-Next, the client can issue SYNC requests for each folder it wants to
populate. Unlike the FOLDERSYNC request, the SYNC request doesn't send
the actual changes back with the 1st synckey. E.g., the first SYNC
request for a mail folder contains some options such as the filtertype
(how far back to sync), bodytype preferences etc... The server sets
those options in the SyncCache for that collection, creates a new
synckey, stores it in the collection's state and sends back the
initial response.
++++ SYNC
+Next, the client can issue SYNC requests for each folder it wants to
populate. Unlike the FOLDERSYNC request, the SYNC request doesn't send
the actual changes back with the 1st synckey. E.g., the first SYNC
request for a mail folder contains some options such as the filtertype
(how far back to sync), bodytype preferences etc... The server sets
those options in the !SyncCache for that collection, creates a new
synckey, stores it in the collection's state and sends back the
initial response.
<code>
2016-11-14T17:04:03+00:00 DEBUG: [10030] I <Synchronize>
2016-11-14T17:04:03+00:00 DEBUG: [10030] I <Folders>
@@ -354,7 +357,18 @@
2016-11-14T17:04:03+00:00 DEBUG: [10030] O <Status>
2016-11-14T17:04:03+00:00 DEBUG: [10030] O 1
2016-11-14T17:04:03+00:00 DEBUG: [10030] O </Status>
2016-11-14T17:04:03+00:00 DEBUG: [10030] O </Folder>
-2016-11-14T17:04:03+00:00 DEBUG: [10030] I </Folders>
-2016-11-14T17:04:03+00:00 DEBUG: [10030] I </Synchronize>
+2016-11-14T17:04:03+00:00 DEBUG: [10030] O </Folders>
+2016-11-14T17:04:03+00:00 DEBUG: [10030] O </Synchronize>
</code>
+
+Now, the next SYNC request for this folder will be sent with the
!SyncKey of {5829ee83-11e8-472d-bc66-272ec0a80160}1. At this point we
know it's the "first sync" i.e., all objects are "new" from the
client's point of view, so we do some optimizations and take
advantage of this. Otherwise, all the future SYNC requests will behave
the same way - the client sends the last !SyncKey it knows about, the
server uses that to get the state of whatever collection we are
syncing, calculates the changes, increments the !SyncKey counter,
sends it all and saves the new state.
+
+The general flow for getting server changes would be something like this:
+
+In ActiveSync_Request_Sync::_handle() --->
+ call into Horde_ActiveSync_Collections::getCollectionChanges() --->
+ which calls into Horde_ActiveSync_State_Base::getChanges() --->
+ which finally calls
Horde_Core_ActiveSync_Driver::getServerChanges() to get the changes.
+
+Once we have the change set in H_AS_Request_Sync::_handle(), we set
it into the exporter object the loop and send each change (see
Horde_ActiveSync_Connector_Exporter_Sync::sendNextChange()).
More information about the commits
mailing list