[Tickets #9306] Personal addressbook not displayed after upgrading

[bugs at horde.org]

BITTE NICHT AUF DIESE NACHRICHT ANTWORTEN. NACHRICHTEN AN DIESE  
E-MAIL-ADRESSE WERDEN NICHT GELESEN.

Ticket-URL: http://bugs.horde.org/ticket/9306
------------------------------------------------------------------------------
  Ticket           | 9306
  Erstellt Von     | michael.rhyner at medianet.ch
  Zusammenfassung  | Personal addressbook not displayed after upgrading
  Warteschlange    | Turba
  Version          | 2.3.5
  Typ              | Bug
  Status           | Unconfirmed
  Priorität       | 2. Medium
  Milestone        | 2.3.6
  Patch            |
  Zuständige      |
------------------------------------------------------------------------------


michael.rhyner at medianet.ch (2010-10-13 12:56) hat geschrieben:

We are a mid-sized ISP in Central Switzerland.

After adding an UTF-8 locale,  e.g. en_US.UTF-8 to the system locales  
so customers would be able to display e-mail messages in UTF-8  
encoding, and private addressbooks of customers having entries  
containing special characters like Umalut, etc, no data will be  
displayed. We have stored data on a mysql 5 server using the  
"iso-8859-1" character set and indicated that fact within  
config/conf.php by telling:
$conf['sql']['charset'] = 'iso-8859-1';

After turning on display_errors within php.ini, the following error  
messages apper:
"Warning: array_values() [function.array-values]: The argument should  
be an array in /var/www/horde/turba/lib/Driver/prefs.php on line 32

Warning: Invalid argument supplied for foreach() in  
/var/www/horde/turba/lib/Driver.php on line 541"

First, we tried to change the setting within turba/config/sources.php:
'charset' => NLS::getCharset()
to
'charset' => 'ISO-8859-1'
But that has changed nothing. After investigating a bit, we've seen  
within the file "turba/lib/Driver/prefs.php, "de-serialization will be  
done after converting the character set from ISO-8859-1 to UTF-8. That  
leads to a failure to unserialize the whole addressbook because the  
byte-length indicated before does not any more correspond to the  
effective byte-length of the respective string if the user had entered  
special characters (ASCII Code > 127).

All other functions are working as expected even using an UTF-8  
enabled system locale, e.g. all other settings stored within the prefs  
backend like Identity, E-Mail settings, etc. So I assume there must be  
something wrong/different with turba prefs storage functions compared  
to the other prefs functions.

For example, within lib/horde/Identity.php starting at line 87, it let  
me assume that at least some tries are being done with and without  
charset conversion so there is a chance to handle both situations  
(conversion needed and not needed before unserialization):
         if (!($this->_identities =  
@unserialize($this->_prefs->getValue('identities', false)))) {
             /* Convert identities from the old format. */
             $this->_identities =  
@unserialize($this->_prefs->getValue('identities'));
         } elseif (is_array($this->_identities)) {
             $this->_identities =  
$this->_prefs->convertFromDriver($this->_identities, NLS::getCharset());
         }

As a workaround, we have to either leave all including the system  
locale to not using UTF-8 and further live with e-mail messages using  
non-iso character sets not being displayed directly (until after  
clicking the link opening a new window in the respective encoding).

Another possibility could be converting all data within the database  
to UTF-8 (and swithing to utf-8 within configuration of course), but  
that leads to even more problems with serialized data within e.g. the  
whole preference settings (no data is displayed e.g. for Identity  
settings, etc.) because data would then be stored as UTF-8 strings,  
but the byte length indicated will be wrong).

Attached you will find horde and turba configuration files.

Thank you in advance for some help or even better a fixed version so  
we are able to use ISO-8859-1 within the databaes, but display content  
as UTF-8 as it seems to be possible.

Regards,
Michael Rhyner
Medianet AG