[imp] Encoding woes with the preview pane in DIMP from Horde 3

[Jan Schneider]

Zitat von Gunnar Wrobel <wrobel at horde.org>:

> Hi!
>
> When removing the various locales on the base system where Horde 3  
> is installed I can break the message preview in DIMP. Pretty much  
> the same thing described in http://bugs.horde.org/ticket/5955
>
> That issue is however closed as not being a bug and I'm not certain  
> I understand the issue completely so I thought I ask here.
>
> Horde 3 is running on Ubuntu 10.04 and I removed the locales with  
> these commands ...
>
> apt-get remove locales
> rm -rf /usr/lib/locale/*.utf8
>
> ... cleared the Horde cache afterwards ....
>
> rm /tmp/cache_*
>
> ... and restarted the apache server:
>
> /etc/init.d/apache restart
>
> For messages that contain German Umlauts (that were displayed fine  
> before) the DIMP preview is empty now. At the same time I get this  
> stack trace in the PHP error log:
>
>
> [25-Aug-2011 17:14:02] PHP Warning:  json_encode() [<a  
> href='function.json-encode'>function.json-encode</a>]: Invalid UTF-8  
> sequence in argument in /www/client/imp/lib/JSON.php on line 86
> [25-Aug-2011 17:14:02] PHP Stack trace:
> [25-Aug-2011 17:14:02] PHP   1. {main}() /www/client/dimp/imp.php:0
> [25-Aug-2011 17:14:02] PHP   2. IMP::sendHTTPResponse()  
> /www/client/dimp/imp.php:767
> [25-Aug-2011 17:14:02] PHP   3. IMP_Serialize_JSON::encode()  
> /www/client/imp/lib/IMP.php:1883
> [25-Aug-2011 17:14:02] PHP   4. json_encode() /www/client/imp/lib/JSON.php:86
>
> As far as I understand it so far the following happens:
>
> When the application gets pushed in the Registry the code tries to  
> set the language environment (NLS::setLang()). This method will try  
> to enable the correct locale but will also check that the locale  
> with the current character set set actually exists. In case the  
> browser supports "utf" the system will try for character set "UTF-8".
>
> Since I removed those locales however NLS::setLang() will fall back  
> to a working character set. In my case the 'ISO-8859-1' charset.  
> After that NLS::getCharset() will always return this value.
>
> Once the system tries to show the message preview it looks at the  
> message converts it to the current character set ('ISO-8859-1') and  
> pipes it into json_encode(). The latter one chokes as it expects  
> UTF-8 and the preview breaks.
>
> I don't think I have the full picture yet so I'm mainly asking if my  
> observation could be correct. If yes, is this to be expected and  
> should the user just ensure that all locales are present in UTF-8?  
> Or is this something that could easily be fixed?

This is the expected behavior. We require UTF-8 support of the system  
for Horde 4 which basically means UTF-8 locales if using translations.  
I was sure this was even mentioned in the prerequisites section of  
INSTALL, or at least in TRANSLATIONS, but obviously not.

Jan.

-- 
Do you need professional PHP or Horde consulting?
http://horde.org/consulting/