[sync] Charset problems when syncing SE W880i calendar

[Bjørn Mork]

I have problems with non us-ascii characters, like æøåÆØÅ, in calendar
events.   The result when syncronising from phone to Kronolith is
truncated entries, ening in a '?'.  

I am using the CVS version of the Horde framework, checked out
2007-08-09. 

bjorn at canardo:~$ grep '$Horde' /usr/local/share/php/SyncML.php 
 * $Horde: framework/SyncML/SyncML.php,v 1.63 2007/06/29 12:58:01 jan Exp $

The database was created by running 
  mysql -uroot -p  <horde/scripts/sql/create.mysql.sql

The kronolith tables were created by running
  mysql -uroot -p  horde < horde/kronolith/scripts/sql/kronolith.mysql.sql


MySQL is using latin1 as default charset:

mysql> status; 
--------------
mysql  Ver 14.12 Distrib 5.0.32, for pc-linux-gnu (i486) using readline 5.2

Connection id:          243740
Current database:       horde
Current user:           root at localhost
SSL:                    Not in use
Current pager:          stdout
Using outfile:          ''
Using delimiter:        ;
Server version:         5.0.32-Debian_7etch1 Debian etch distribution
Protocol version:       10
Connection:             Localhost via UNIX socket
Server characterset:    latin1
Db     characterset:    latin1
Client characterset:    latin1
Conn.  characterset:    latin1
UNIX socket:            /var/run/mysqld/mysqld.sock
Uptime:                 40 days 13 hours 54 min 4 sec

Threads: 2  Questions: 21618646  Slow queries: 0  Opens: 47454  Flush tables: 1  Open tables: 62  Queries per second avg: 6.166
--------------



Example: This event had the original title 'Test Bjørn' on the phone:

mysql> select * from kronolith_events where event_uid = '20070815112047.zc3g8sokvpw at www.example.com'\G
*************************** 1. row ***************************
           event_id: ebc816dbd4b8c5815e0399be24592e4b
          event_uid: 20070815112047.zc3g8sokvpw at www.example.com
        calendar_id: luser
   event_creator_id: luser
  event_description: 
     event_location: 
       event_status: 2
    event_attendees: a:0:{}
     event_keywords: 
   event_exceptions: NULL
        event_title: Test Bj?
     event_category: 
    event_recurtype: 0
event_recurinterval: NULL
    event_recurdays: NULL
 event_recurenddate: NULL
   event_recurcount: NULL
        event_start: 2007-08-15 12:00:00
          event_end: 2007-08-15 12:30:00
        event_alarm: 15
     event_modified: 1187169645
      event_private: 0
1 row in set (0.01 sec)


Looks like an utf8/iso8859-1 confusion to me. The data from the phone
seems to be quoted-printable iso8859-1, and the truncated result
indicates that it has been interpreted as utf8.

Note that syncronising the address book does *not* show these problems.
Characters are converted as they should there.

I'm attaching the sync log and the relevant wbxml file from the
client. The wbxml file has been converted to xml using 'wbxml2xml -i 4'.  
Actual URL and IMEI have been replaced by dummy values in both files.



Bjørn

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: syncml_log_conv.txt
Url: http://lists.horde.org/archives/sync/attachments/20070816/ad35e16e/attachment.txt