[imp] Encoding woes with the preview pane in DIMP from Horde 3

[Gunnar Wrobel]

Quoting Jan Schneider <jan at horde.org>:

> Zitat von Gunnar Wrobel <wrobel at horde.org>:
>
>> Hi!
>>
>> When removing the various locales on the base system where Horde 3  
>> is installed I can break the message preview in DIMP. Pretty much  
>> the same thing described in http://bugs.horde.org/ticket/5955
>>
>> That issue is however closed as not being a bug and I'm not certain  
>> I understand the issue completely so I thought I ask here.
>>
>> Horde 3 is running on Ubuntu 10.04 and I removed the locales with  
>> these commands ...
>>
>> apt-get remove locales
>> rm -rf /usr/lib/locale/*.utf8
>>
>> ... cleared the Horde cache afterwards ....
>>
>> rm /tmp/cache_*
>>
>> ... and restarted the apache server:
>>
>> /etc/init.d/apache restart
>>
>> For messages that contain German Umlauts (that were displayed fine  
>> before) the DIMP preview is empty now. At the same time I get this  
>> stack trace in the PHP error log:
>>
>>
>> [25-Aug-2011 17:14:02] PHP Warning:  json_encode() [<a  
>> href='function.json-encode'>function.json-encode</a>]: Invalid  
>> UTF-8 sequence in argument in /www/client/imp/lib/JSON.php on line 86
>> [25-Aug-2011 17:14:02] PHP Stack trace:
>> [25-Aug-2011 17:14:02] PHP   1. {main}() /www/client/dimp/imp.php:0
>> [25-Aug-2011 17:14:02] PHP   2. IMP::sendHTTPResponse()  
>> /www/client/dimp/imp.php:767
>> [25-Aug-2011 17:14:02] PHP   3. IMP_Serialize_JSON::encode()  
>> /www/client/imp/lib/IMP.php:1883
>> [25-Aug-2011 17:14:02] PHP   4. json_encode()  
>> /www/client/imp/lib/JSON.php:86
>>
>> As far as I understand it so far the following happens:
>>
>> When the application gets pushed in the Registry the code tries to  
>> set the language environment (NLS::setLang()). This method will try  
>> to enable the correct locale but will also check that the locale  
>> with the current character set set actually exists. In case the  
>> browser supports "utf" the system will try for character set "UTF-8".
>>
>> Since I removed those locales however NLS::setLang() will fall back  
>> to a working character set. In my case the 'ISO-8859-1' charset.  
>> After that NLS::getCharset() will always return this value.
>>
>> Once the system tries to show the message preview it looks at the  
>> message converts it to the current character set ('ISO-8859-1') and  
>> pipes it into json_encode(). The latter one chokes as it expects  
>> UTF-8 and the preview breaks.
>>
>> I don't think I have the full picture yet so I'm mainly asking if  
>> my observation could be correct. If yes, is this to be expected and  
>> should the user just ensure that all locales are present in UTF-8?  
>> Or is this something that could easily be fixed?
>
> This is the expected behavior.

Need to ask again as it looks as though I have to provide at least  
some kind of fix for the broken preview in Horde 3: I would assume  
that for the JSON based preview it is sufficient if I check if  
NLS::getCharset() returns a non-"UTF-8" value. If it does I would  
return some information like "The server does not support rendering  
the preview for your locale. Contact your system administrator." Does  
that sound okay or is it completely off? I didn't look at the code  
again.

Cheers,

Gunnar

> We require UTF-8 support of the system for Horde 4 which basically  
> means UTF-8 locales if using translations. I was sure this was even  
> mentioned in the prerequisites section of INSTALL, or at least in  
> TRANSLATIONS, but obviously not.
>
> Jan.
>
> -- 
> Do you need professional PHP or Horde consulting?
> http://horde.org/consulting/
>
> -- 
> IMP mailing list
> Frequently Asked Questions: http://horde.org/faq/
> To unsubscribe, mail: imp-unsubscribe at lists.horde.org

-- 
Core Developer
The Horde Project

e: wrobel at horde.org
t: +49 700 6245 0000
w: http://www.horde.org

pgp: 9703 43BE
tweets: http://twitter.com/pardus_de
blog: http://log.pardus.de