[commits] [Wiki] changed: ActiveSync

[Michael Rubinsky]

mrubinsk  Wed, 13 Jun 2012 21:14:54 +0000

Modified page: http://wiki.horde.org/ActiveSync
New Revision:  145
Change log:  Some tweaks, corrections, and back patting

@@ -1,25 +1,25 @@
  [[toc]]

  + !ActiveSync

-Horde 4 has support for allowing Horde to sync with mobile devices  
using the mobile device's support for Microsoft Exchange/!ActiveSync.  
The code that Horde's !ActiveSync implementation uses for handling the  
protocol communication is based in part on the Z-Push library  
(http://z-push.sourceforge.net).
+Support for allowing Horde to sync with mobile devices using support  
for Microsoft Exchange/!ActiveSync. This functionality has been  
available and stable since Horde 4.

  ++ People

  [MichaelRubinsky Michael Rubinsky]

  ++ Description

-The Horde !ActiveSync implementation provides over the air  
synchronization of contacts, calendar, task, and Email (Horde 5) data  
to devices such as the iPhone/iPod Touch, Android and any other system  
supporting !ActiveSync. Horde 4's implementation supports Microsoft  
!ActiveSync protocol versions up to 2.5 -the version implemented by  
Microsoft Exchange 2003 while Horde 5 will support up to 12.0/12.1 -  
the version implemented by Microsoft Exchange 2007sp1.
+The Horde !ActiveSync implementation provides over the air  
synchronization of contacts, calendar, task, and (since Horde 5) Email  
data to devices such as the iPhone/iPod Touch, Android and any other  
system supporting Exchange !ActiveSync. Horde 4's implementation  
supports Microsoft !ActiveSync protocol versions up to 2.5 -the  
version implemented by Microsoft Exchange 2003 while Horde 5 supports  
up to 12.1 - the version implemented by Microsoft Exchange 2007sp1.

  ++ Server Setup

  !ActiveSync support is stable, but there are still devices on which  
it has not been tested yet. If you have used a device with Horde's  
!ActiveSync support that is not listed below, please feel free to let  
us know how it went.

-To activate the server, it needs to be enabled in Horde's  
configuration, on the !ActiveSync tab. It's already enabled by  
default. The SQL tables that horde uses are created as usual from the  
Horde configuration screen.
+To activate the server, it needs to be enabled in Horde's  
configuration, on the !ActiveSync tab. The SQL tables that horde uses  
are created as usual from the Horde configuration screen.

-Next, you will need to configure your webserver to redirect the URL  
/Microsoft-Server-ActiveSync to your horde/rpc.php file. How you do  
this depends on your webserver and it's configuration. For Apache,  
something like:
+You will also need to configure your webserver to redirect the URL  
/Microsoft-Server-ActiveSync to your horde/rpc.php file. How you do  
this depends on your webserver and it's configuration. For Apache,  
something like:
  <code>
  Alias /Microsoft-Server-ActiveSync /var/www/horde/rpc.php
  </code>

@@ -48,21 +48,22 @@
  <code>
  alias.url = ("/Microsoft-Server-ActiveSync" => "/var/www/horde/rpc.php",
               "/autodiscover/autodiscover.xml" => /var/www/horde/rpc.php");
  </code>
+Note that Autodiscovery support is VERY flaky in most clients that we  
have tested.

  ++ Application Configuration
  No additional steps are normally necessary for synchronization of  
the supported applications. However, each application that supports  
synchronization also has a user preference to determine which shares  
will be synchronized. For example, in Kronolith the user's default  
calendar is *always* synchronized, but the user can choose to add any  
additional calendars he/she owns. Whenever the user changes one of  
these settings, the device is always automatically re-paired with the  
server to pick up the change.

  Users can also view all their paired !ActiveSync devices by visiting  
their //!ActiveSync Devices// preferences. This is located within  
Horde's //Global Preferences//. From here, it is possible to force a  
complete re-sync, or to request a remote wipe of a provisioned device  
(see below).

  ++ Email Support

-Email synchronization has been added in Horde 5. Since !ActiveSync  
email support is inferior to many native email clients, this  
functionality is disabled by default. If you would like to enable  
Email support, you need to select this in the !ActiveSync tab of  
Horde's configuration screen.
+Email synchronization has been added in Horde 5. Since in some  
installs this could increase server load or may be otherwise  
undesirable, it is possible to deactivate email support via Horde's  
configuration, on the !ActiveSync tab.

-!ActiveSync email support requires an IMAP server, not POP3. It will  
use the same server that IMP is configured to connect to. It is  
recommended that this server support the QRESYNC server extension for  
performance reasons, though it will work without this. It can also  
help performance if an IMAP proxy is used.
+!ActiveSync email support requires an IMAP server POP3 is not  
supported. It will use the same server that IMP is configured to  
connect to. It is recommended that this server support the QRESYNC  
server extension for performance reasons, though it will work without  
this. It can also help performance if an IMAP proxy is used.

-The only flag supported by !ActiveSync is the //seen// flag. Flag  
changes will be synchronized, but flag changes alone will not trigger  
a SYNC (again, for performance reasons). The only thing that will  
trigger a SYNC is the arrival of a new message (technically, an  
increase in the NEXTUID value). Once this SYNC is triggered though,  
all message changes are taken into account.
+The only flags supported by !ActiveSync are the //seen// and  
//flagged for follow up// flags. Flag changes will be synchronized,  
but flag changes alone will **not** trigger a SYNC for performance  
reasons. The only thing that will trigger a SYNC is the arrival of a  
new message (technically, an increase in the NEXTUID value). Once this  
SYNC is triggered though, all message changes are taken into account -  
including any flag changes.

  Since !ActiveSync does not support the //deleted// flag, messages in  
a mailbox with this flag are ignored when syncing. Deleting a message  
on the device will do one of two things; If the user has enabled a  
Trash mailbox then the message will be moved to that mailbox.  
Otherwise, the message is immediately expunged. This is in accordance  
with the !ActiveSync protocol specs.

  Forwarding a message will always attempt to put the main message  
text body in-line and keep any original attachments. It seems that a  
number of devices cannot view message/rfc822 attachments, so any  
messages that have been forwarded as an attachment may not be viewable  
in the !ActiveSync mail client.
@@ -72,12 +73,12 @@
  Some devices attempt to use Exchange's Autodiscovery service to make  
it easier for both the user to setup the account and for the  
administrator to make drastic changes like moving the server to a new  
URL. Horde attempts to support this as best it can. For this to work,  
you must create the URL alias as described above, and Horde must be  
able to figure out the Horde username based on the email address the  
user provided to the device. The configuration screen provides  
multiple options for this. In the worst case, if Horde cannot  
authenticate based on the provided information from the Autodiscover  
request, the device will fall back to requiring manual configuration.

  ++ !ActiveSync Versions

-Horde 5 adds support for !ActiveSync version 12.0 - the version  
shipped with Exchange Server 2007. This adds HTML email support, flag  
or followup, more atomic policy settings, additional search sources,  
local wipe rules, and WBXML based provisioning (instead of the XML  
used in 2.5). There are a few things to mention regarding 12.0 support:
+Horde 5 adds support for !ActiveSync versions 12.0 and 12.1 - the  
version shipped with Exchange Server 2007 and 2007sp1. This adds among  
other things: HTML email support, flagged for followup, more atomic  
policy settings, additional search sources, local wipe rules, and  
WBXML based provisioning (instead of the XML used in 2.5). There are a  
few things to mention regarding 12.0 support:

-* It seems some device, most notably older Android devices that will  
not automatically detect the new version support - even if the device  
is removed from Horde. You must actually remove the account from the  
device and configure it again. So, if you are not seeing 12.0 only  
features, be sure to try re-adding the account to the device.
-* It seems that iOS devices do NOT support 12.0. The device will  
communicate using version 2.5 with a 12.0 server. The next version  
they support is 12.1 - what ships with Exchange Server 2007sp1.  
Support for this version is planned for an upcoming Horde 5.x release.
+* It seems some devices, most notably older Android devices, will not  
automatically detect the new version support - even if the device is  
removed from Horde. You must actually remove the account from the  
device and configure it again. So, if you are not seeing 12.0/12.1  
only features, be sure to try re-adding the account to the device.
+* It seems that iOS devices do NOT support 12.0. The device will  
communicate using version 2.5 with a 12.0 server. The next version  
they support is 12.1 - what ships with Exchange Server 2007sp1. So, if  
you are seeing 2.5 only features on an iOS device, be sure the Horde  
server has been configured to support 12.1. If it has, then see the  
first point above.

  ++ Administration
  Administrators can view all of the !ActiveSync devices paired with  
the server. This is the //!ActiveSync Devices// link located under the  
Administration menu. From here an administrator can request a remote  
wipe, or force a re-provisioning of any device.

@@ -91,9 +92,9 @@
  Clicking //Wipe// in the Horde interfaces for device management  
flags the server to send the wipe command to the device the next time  
it synchronizes. The next time the device attempts to request a  
command other then PING or OPTIONS, it will be wiped. The !ActiveSync  
preference page shows the status of all the user's devices. If the  
status is listed as //Pending//, and you wish to cancel the wipe  
request, you may do this by clicking the //Cancel Wipe// button. You  
should see the status be reset to //Provisioned//. After it is wiped,  
the status will be shown as //Wiped//, if you wish to allow the device  
to connect to your server again, you need to explicitly remove the  
device as a sync partner by clicking the //Remove// button. If you do  
not remove this entry, the device will continue to be wiped each time  
it reconnects to the server.

  ++ What works

-Contacts, Calendar and Task syncing are working. Note that not all  
devices support Tasks. Of the tested devices, iOS (versions < 5.0) and  
Android are lacking native Task applications. The !TouchDown client  
and Windows Mobile both support Tasks.
+Contacts, Calendar, Task and Email syncing are all working. Note that  
not all devices support Tasks. Of the tested devices, iOS (versions <  
5.0) and Android are lacking native Task applications. The !TouchDown  
client, Moxier Mail,  and Windows Mobile both support Tasks.

  Client searching of an "Exchange Global Address List" (GAL) is  
supported, provided that Turba (or other application providing the  
//contacts// api) is configured to provide a GAL. See the Turba  
configuration for details.

  Provisioning along with remote wipe is also working on devices that  
actually support it.
@@ -121,13 +122,13 @@
  || Windows Phone 7 || 7.0 || ? || ? || Contacts, Calendars || 2.5 ||

  ++ Setting up the device

-It's beyond the scope of this page to go into detail for each  
individual device. In general, you will need to create a new account  
on the device. The account type should be something like //Microsoft  
Exchange// or //!ActiveSync//. Some devices use //Corporate//. You  
will need to enter your normal Horde username and password in the  
appropriate fields. In the field for the server address, you should  
enter the **root** of the webserver or virtual host that hosts Horde.  
For example, if you host horde at http://host.example.com/horde then  
you should enter //host.example.com//. You can ignore any reference to  
a //domain// entry.
+It's beyond the scope of this page to go into detail for each  
individual device. In general, you will need to create a new account  
on the device. The account type should be something like //Microsoft  
Exchange// or //!ActiveSync//. Some devices use //Corporate//. You  
will need to enter your normal Horde username and password in the  
appropriate fields. In the field for the server address, you should  
enter the **root** of the webserver or virtual host that hosts Horde.  
For example, if you host horde at http://host.example.com/horde then  
you should enter //host.example.com//. You can ignore any reference to  
a //domain// entry. If the device **requires** the domain entry (some  
Windows Mobile devices do this) you may safely enter any value.

  A special note for the iPhone/iPod (and possibly others) - if you do  
not use a SSL enabled site you may receive errors about not being able  
to find the !ActiveSync server. If this happens, just continue, or  
save, or whatever your option is to continue.  On the iPhone, after  
everything is completely set up, you must go back into the account  
settings and disable SSL.

-After the connection particulars are entered, you should choose to  
enable the folders that you want sync'd. Contacts, Calendar and Tasks  
are supported, but your device also needs to support the requested  
folders.
+After the connection particulars are entered, you should choose to  
enable the folders that you want sync'd.

  ++ What to do if you have problems (or How to help us help you)

  **If you are not even able to get past the initial setup page on  
your phone**: you should first check to be sure you do not have SSL  
enabled on the phone when you're server is not serving SSL. The  
iPhone/iPod will not let you turn this off until after you save the  
configuration, so you must continue through all the errors and go back  
into the settings to disable SSL. You should also make sure that you  
have not enabled Provisioning support if your phone does not support it.
@@ -167,9 +168,9 @@
  * "Enable Exchange SD card logging" - If checked, puts all of the  
Exchange related logging onto SD card in a file named emaillog.txt.    
This file grows continuously until deleted (or the checkbox is  
unchecked).  This can be helpful in finding issues that are sporadic,  
as the log is essentially unlimited in size.

  ++ Horde_ActiveSync vs. Z-Push

-Horde_ActiveSync was based on Z-Push. The code that handles the  
protocol level is essentially the same, though it has been heavily  
refactored and cleaned.
+The protocol handling in Horde_ActiveSync was based on Z-Push. The  
code that handles the protocol level is essentially the same, though  
it has been heavily refactored and cleaned.

  Z-Push comes out of the box with a number of backends. The only one  
that is really //fully// functional is the "ICS" backend which  
connects to a Zarafa server. In addition to the ICS backend, Z-Push  
also provides a number of other backends - all of which extend what  
they call the "Diff" backend.

  The diff backend is a very inefficient way of determining what needs  
to be synched. It uses file based storage - depending on the Z-Push  
version it uses either a single file or a directory of files for each  
device. These files contain, along with some basic device state  
information, a list of every UID that is on the device.
@@ -185,22 +186,22 @@
  * A !SearchLdap backend that - from what I can tell - is used as a  
Global Address Book source for searching for individual contacts (not  
for syncing the entire address book).

  As of the time Horde_ActiveSync was written, you could only use  
//one// backend at a time - so, unless you were using Zarafa (or maybe  
Kolab) you could sync email //or// contacts. Since then they have  
started a "combined" backend that is supposed to wrap any number of  
backends. Last I checked it wasn't complete yet.

-The main differences between 1.5.x and 2 are the versions of EAS that  
are supported. 1.5.x supports only up to Exchange 2003sp1 (same as  
Horde 4). Version 2 is supposedly going to support up to EAS 14  
(Exchange 2010) - though I believe only up to 12.1 (Exchange 2007) is  
currently working.
+The main differences between 1.5.x and 2 are the versions of EAS that  
are supported. 1.5.x supports only up to Exchange 2003sp1 (same as  
Horde 4). Version 2 is supposedly going to support up to EAS 14  
(Exchange 2010) - though I believe only up to 12.1 (Exchange 2007) is  
working...and I don't believe that is even fully functional.

  +++The main differences

  Some of these are specific to using Horde data as a backend to  
Horde_ActiveSync:

-# Modularity. Separate classes for maintaining device state and for  
obtaining message diffs. If not using Horde as a backend, all that  
would theoretically be needed is to write a new class that extends  
Horde_ActiveSync_Driver_Base. This allows the backend, itself, to  
determine the best way to calculate diffs...like using Horde's History  
system. Support is also built in to allow using a different method of  
determining diffs based on the collection we are syncing. So, for  
example, if we ever implement email we would need a way of determining  
diffs that is not based on on the History system. This implementation  
allows us to easily add a new diff system and use multiple diff  
systems at once e.g., use History for contacts/calendar/tasks and some  
other diff system for email. In Z-Push, diff generation and state  
management are tightly coupled.
-# Efficiency. The history based diff engine is orders of magnitude  
more efficient than using Z-Push's file based diff backend.
+# Modularity. Separate classes for maintaining device state and for  
obtaining message diffs. If not using Horde as a backend, all that is  
needed is to write a new class that extends  
Horde_ActiveSync_Driver_Base. This allows the backend, itself, to  
determine the best way to calculate diffs...like using Horde's History  
system. In Z-Push, diff generation and state management are tightly  
coupled.
+# Efficiency. The history based diff engine is orders of magnitude  
more efficient than Z-Push's file based diff backend.
  # Unless using Zarafa, Z-Push contained no message-specific logic.  
For example, no code for dealing with appointment related issues such  
as timezones, recurrence series etc. Horde_ActiveSync contains support  
for these things out of the box.
-# At least at the time of writing, Horde allowed better configuration  
of things such as heartbeat/timeout intervals. We also actually allow  
configuring available security policies. Z-Push had basic support for  
provisioning and for turning on or off the requirement for a device  
PIN - but contained no facility for configuring any of the other  
options.  Also, if using Horde_ActiveSync as part of a typical Horde  
install you get all the ease of configuration that our administration  
interface provides. I think some of this has improved as part of  
implementing more recent EAS versions in Z-Push.
-# I don't have hard data on device compatibility as it compares to  
Z-Push, but I do know that we have fixed some device specific issues  
in the past that - at least at the time - were not working with  
Z-Push. Certain Nokia devices come to mind that use !MfE. Z-Push lists  
these as "unknown" compatibility but they work, at least for  
contacts/calendar with Horde.
-# Email sync with Horde_ActiveSync uses the vastly more efficient  
Horde_Imap_Client library, Z-Push uses the c-client PHP extension.
-# Horde's email support is more complete than Z-Push's IMAP based  
implementation - with support for version 12.0 style email store  
searching, follow-up flag synchronization, and more.
-# Horde_ActiveSync currently supports up to Exchange 2007 - Z-Push  
has code in development that works to some extent with 2007 and plans  
to look at supporting up to 2010.
+# At least at the time of writing, Horde allowed better configuration  
of things such as heartbeat/timeout intervals. We also actually allow  
configuring available security policies. Z-Push had basic support for  
provisioning and for turning on or off the requirement for a device  
PIN - but contained no facility for configuring any of the other  
options without editing code.  Also, if using Horde_ActiveSync as part  
of a typical Horde install you get all the ease of configuration that  
our administration interface provides.
+# I don't have hard data on device compatibility as it compares to  
Z-Push, but I do know that we have fixed some device specific issues  
in the past that - at least at the time of this writing - were not  
working with Z-Push. Certain Nokia devices come to mind that use !MfE.  
Z-Push lists these as "unknown" compatibility but they work, at least  
for contacts/calendar with Horde. Remote wiping iOS devices is also  
problematic in Z-Push.
+# Email sync with Horde_ActiveSync uses the vastly more efficient  
Horde_Imap_Client library and takes full advantage of IMAP servers  
with QRESYNC capabilities. Z-Push uses the very poor performing  
c-client PHP extension.
+# Horde's email support is more complete than Z-Push's IMAP based  
implementation - with support for version 12.0 style email store  
searching, follow-up flag synchronization, S/Mime signatures and  
encryption, and more.
+# Horde_ActiveSync currently supports almost the full feature set up  
to Exchange 2007sp1 - Z-Push has code in development that works to  
some extent with 2007 and plans to look at supporting up to 2010.
  # Horde_ActiveSync supports multiple users per device - when the  
device supports it. AFAIK, Z-Push only supports a single user account  
per device.

  ++ Resources