[sync] Palm Tungsten C, Synthesis, Results, Patches, and Progress
Nathan Sharp
horde at nps.parcellsharp.net
Wed Jun 8 05:00:07 PDT 2005
I am not a little bit excited to see a good open source calendar project
finally doing something about syncing, and am hoping to see it working
before too terribly long. I've been playing with the CVS HEAD (Jun 2,
05) and a Palm Tungsten C and have some results to post (for the
information of others who might see the same behavior), a couple of
patches, and some questions that I still haven't got working. Hopefully
this will be generally helpful, sorry to be so long.
1) I tried using the Gnome multisync tool but was generally not real
pleased with the performance or stability of the code. This isn't to
say that I am completely given up on that tool, but just haven't had
time to muck with it any further. (http://multisync.sourceforge.net/)
2) My most luck has come from the commercial synthesis client
(http://www.synthesis.ch/). However, when I first started trying, all I
could get it to do was give me "Invalid Packet". The server reported
into the horde.log file that the client was not authorizing to the
server (authorized= was blank). The solution to this turned out to be
to compile and install libwbxml (http://libwbxml.aymerick.com/) as the
wbxml implementation in Horde didn't correctly convert the Cred
section. I also had to change the hardcoded paths to the command line
tools in <pearinstall>/XML/WBXML/Encoder.php and Decoder.php.
3) I found a couple of bugs in the Encoder.php and Decoder.php. First,
they open up the third pipe for reading, which is technically
incorrect. It seems to have the side-effect of working as if you opened
up /dev/null, which perhaps is a good thing since you aren't reading the
stderr anyhow, although I was having some strange things happening until
I made it write to a log file. In my patch below, I open up a hardcoded
log file for writing. More importantly is that writing to the stdin
entirely, then reading from the stdout is prone to deadlock, and in fact
when my client tried to send the first 67 items (~35k of xml), it hung.
My patch below shows me writing the input to a temp file and telling the
sub-process to read from it, instead of from us. This makes
communication work correctly for me.
--- /var/www/html/horde2/framework/XML_WBXML/WBXML/Encoder.php
2005-06-02 21:57:18.000000000 -0400
+++ Encoder.php 2005-06-08 07:07:25.387996818 -0400
@@ -67,7 +67,7 @@
* Use wbxml2xml from libwbxml.
* @var string $_xml2wbxml
*/
- var $_xml2wbxml = '/usr/bin/xml2wbxml';
+ var $_xml2wbxml = '/usr/local/bin/xml2wbxml';
/**
* Arguments to pass to xml2wbxml.
@@ -92,17 +92,23 @@
function encode($xml)
{
if (!empty($this->_xml2wbxml) &&
is_executable($this->_xml2wbxml)) {
+ //Nathan P Sharp: Just calling a sub-process and dumping
data to the
+ //stdin, then sequentially reading the stdout is prone to
deadlock.
+ //Here I dump the input to a temporary file to prevent it.
+ $inputFile = tempnam("","");
+ $handle = fopen($inputFile, "wb");
+ fwrite($handle, $xml);
+ fclose($handle);
+
$descriptorspec = array(
- 0 => array('pipe', 'r'),
+ 0 => array('file',$inputFile, 'r'),
1 => array('pipe', 'w'),
- 2 => array('pipe', 'r'),
+ 2 => array('file', '/tmp/sync/xml2wbxml.log', 'w'),
);
$xml2wbxml = proc_open($this->_xml2wbxml . ' ' .
$this->_xml2wbxml_args,
$descriptorspec, $pipes);
if (is_resource($xml2wbxml)) {
- fwrite($pipes[0], $xml);
- fclose($pipes[0]);
// Grab the output of xml2wbxml.
$wbxml = '';
@@ -113,8 +119,10 @@
$rv = proc_close($xml2wbxml);
+ unlink($inputFile);
return $wbxml;
} else {
+ unlink($inputFile);
return PEAR::raiseError('xml2wbxml failed');
}
} else {
--- /var/www/html/horde2/framework/XML_WBXML/WBXML/Decoder.php
2005-06-02 21:57:17.000000000 -0400
+++ Decoder.php 2005-06-08 07:17:22.073777470 -0400
@@ -73,7 +73,7 @@
*
* @var string
*/
- var $_wbxml2xml = '/usr/bin/wbxml2xml';
+ var $_wbxml2xml = '/usr/local/bin/wbxml2xml';
/**
* Arguments to pass to wbxml2xml.
@@ -114,17 +114,26 @@
// Figure out if we're going to use wbxml2xml to do the
// conversion, or do it all in PHP code.
if (!empty($this->_wbxml2xml) &&
is_executable($this->_wbxml2xml)) {
+ //Nathan P Sharp: Just calling a sub-process and dumping
data to the
+ //stdin, then sequentially reading the stdout is prone to
deadlock.
+ //Here I dump the input to a temporary file to prevent it.
+ $inputFile = tempnam("","");
+ $handle = fopen($inputFile, "wb");
+ fwrite($handle, $wbxml);
+ fclose($handle);
+
$descriptorspec = array(
- 0 => array('pipe', 'r'),
+ 0 => array('file', $inputFile, "rb"),
1 => array('pipe', 'w'),
- 2 => array('pipe', 'r'),
+ 2 => array('file', '/tmp/sync/wbxml.log', 'w'),
);
$wbxml2xml = proc_open($this->_wbxml2xml . ' ' .
$this->_wbxml2xml_args,
$descriptorspec, $pipes);
if (is_resource($wbxml2xml)) {
- fwrite($pipes[0], $wbxml);
- fclose($pipes[0]);
+ //fwrite($pipes[0], $wbxml);
+ //fclose($pipes[0]);
+
// Grab the output of wbxml2xml.
$xml = '';
@@ -133,10 +142,12 @@
}
fclose($pipes[1]);
+
$rv = proc_close($wbxml2xml);
-
+ unlink($inputFile);
return $xml;
} else {
+ unlink($inputFile);
return PEAR::raiseError('wbxml2xml failed');
}
} else {
4) It seems somewhat sensitive to errors in the database and
environment, so between each test, I got to using the following set of
commands to clear everything out:
I use MySQL for everything, including vfs and the horde datatree.
Here are my SQL commands:
delete from kronolith_events where calendar_id='nsharp';
delete from horde_datatree where group_uid='syncml';
delete from horde_datatree_attributes using
horde_datatree_attributes left join horde_datatree on
horde_datatree_attributes.datatree_id=horde_datatree.datatree_id where
horde_datatree.datatree_id IS NULL;
I would then blow away all my session data (rm -f /tmp/sess_*, but it
depends on how your sessions are set up)
There were also a ton of entries in horde_datatree with
group_uid="horde.history". They didn't seem harmful, but at some point
I deleted them (had >2000 of them)
It is also useful to clear out horde.log and /tmp/sync to get a
better feel for what is from this run and what isn't.
5) At some point I had serious trouble with horde running what seemed
like an infinite loop (although perhaps it was just really slow
performance). I suspect it may have been partly due to not fully
cleaning up the database and session data. Since then synthesis has
gone into "difference" mode instead of "send everything" mode, and I
haven't tried a full re-sync yet. Eventually I'd like to get all 2k
items synced, but for now just getting it working at all is first.
6) In order to cull down the number of items, I attempted to set
synthesis to only sync the last 7 days and next 14. I ran into the same
problem that the poster yesterday mentioned, but was able to get around
it with this cheap and dirty fix; I simply cull off anything after the ?
in the database name. This is probably not technically correct, but
works for me:
--- /var/www/html/horde2/framework/SyncML/SyncML/Command/Alert.php
2005-06-02 21:56:42.000000000 -0400
+++ Alert.php 2005-06-08 07:31:50.621785484 -0400
@@ -282,7 +282,7 @@
if ($this->_Stack[2] == 'Source') {
$this->_sourceLocURI = trim($this->_chars);
} else if ($this->_Stack[2] == 'Target') {
- $this->_targetLocURI = trim($this->_chars);
+ $this->_targetLocURI = ereg_replace("\\?.*$","",
trim($this->_chars));
}
}
break;
--- /var/www/html/horde2/framework/SyncML/SyncML/Command/Sync.php
2005-06-02 21:56:43.000000000 -0400
+++ Sync.php 2005-06-08 07:32:59.197964272 -0400
@@ -145,7 +145,7 @@
if ($this->_Stack[1] == 'Source') {
$this->_sourceURI = trim($this->_chars);
} else if ($this->_Stack[1] == 'Target') {
- $this->_targetURI = trim($this->_chars);
+ $this->_targetURI = ereg_replace("\\?.*$","",
trim($this->_chars));
}
}
break;
Note that the client still seems to send all items, it was right about
then that suddenly the client stopped trying to do a full initial
slow-sync. I got a random assortment of 67 events from my list of 2k,
and now it only tries to send new or changed items. Like I said, I
haven't tried a re-sync all yet.
7) My last problem is that when the server tries to send an event to the
Palm, the palm sends a "format not supported error" (I forgot to write
down the http code). My guess is that it doesn't support vcalendar
version 2 format (the client always sends v1 events). Looking at the
code it seems like the right thing to do is to write a Device in the
Devices directory for Synthesis that does the conversion. I haven't had
time to mess with this yet.
Thanks for the great code!
Nathan
More information about the sync
mailing list