[imp] Encoding woes with the preview pane in DIMP from Horde 3

[Jan Schneider]

Zitat von Gunnar Wrobel <wrobel at horde.org>:

> Quoting Jan Schneider <jan at horde.org>:
>
>> Zitat von Gunnar Wrobel <wrobel at horde.org>:
>>
>>> Hi!
>>>
>>> When removing the various locales on the base system where Horde 3  
>>> is installed I can break the message preview in DIMP. Pretty much  
>>> the same thing described in http://bugs.horde.org/ticket/5955
>>>
>>> That issue is however closed as not being a bug and I'm not  
>>> certain I understand the issue completely so I thought I ask here.
>>>
>>> Horde 3 is running on Ubuntu 10.04 and I removed the locales with  
>>> these commands ...
>>>
>>> apt-get remove locales
>>> rm -rf /usr/lib/locale/*.utf8
>>>
>>> ... cleared the Horde cache afterwards ....
>>>
>>> rm /tmp/cache_*
>>>
>>> ... and restarted the apache server:
>>>
>>> /etc/init.d/apache restart
>>>
>>> For messages that contain German Umlauts (that were displayed fine  
>>> before) the DIMP preview is empty now. At the same time I get this  
>>> stack trace in the PHP error log:
>>>
>>>
>>> [25-Aug-2011 17:14:02] PHP Warning:  json_encode() [<a  
>>> href='function.json-encode'>function.json-encode</a>]: Invalid  
>>> UTF-8 sequence in argument in /www/client/imp/lib/JSON.php on line  
>>> 86
>>> [25-Aug-2011 17:14:02] PHP Stack trace:
>>> [25-Aug-2011 17:14:02] PHP   1. {main}() /www/client/dimp/imp.php:0
>>> [25-Aug-2011 17:14:02] PHP   2. IMP::sendHTTPResponse()  
>>> /www/client/dimp/imp.php:767
>>> [25-Aug-2011 17:14:02] PHP   3. IMP_Serialize_JSON::encode()  
>>> /www/client/imp/lib/IMP.php:1883
>>> [25-Aug-2011 17:14:02] PHP   4. json_encode()  
>>> /www/client/imp/lib/JSON.php:86
>>>
>>> As far as I understand it so far the following happens:
>>>
>>> When the application gets pushed in the Registry the code tries to  
>>> set the language environment (NLS::setLang()). This method will  
>>> try to enable the correct locale but will also check that the  
>>> locale with the current character set set actually exists. In case  
>>> the browser supports "utf" the system will try for character set  
>>> "UTF-8".
>>>
>>> Since I removed those locales however NLS::setLang() will fall  
>>> back to a working character set. In my case the 'ISO-8859-1'  
>>> charset. After that NLS::getCharset() will always return this value.
>>>
>>> Once the system tries to show the message preview it looks at the  
>>> message converts it to the current character set ('ISO-8859-1')  
>>> and pipes it into json_encode(). The latter one chokes as it  
>>> expects UTF-8 and the preview breaks.
>>>
>>> I don't think I have the full picture yet so I'm mainly asking if  
>>> my observation could be correct. If yes, is this to be expected  
>>> and should the user just ensure that all locales are present in  
>>> UTF-8? Or is this something that could easily be fixed?
>>
>> This is the expected behavior.
>
> Need to ask again as it looks as though I have to provide at least  
> some kind of fix for the broken preview in Horde 3: I would assume  
> that for the JSON based preview it is sufficient if I check if  
> NLS::getCharset() returns a non-"UTF-8" value. If it does I would  
> return some information like "The server does not support rendering  
> the preview for your locale. Contact your system administrator."  
> Does that sound okay or is it completely off? I didn't look at the  
> code again.

Looking at the code, it actually shouldn't happen that we provide  
non-UTF-8 translations at all, because we don't fall back to the  
language's available charset anymore. But maybe this happened because  
we still set the environment variables to the language and charset, no  
matter if that locale exists.
Try if what I committed improves the behavior.

>> We require UTF-8 support of the system for Horde 4 which basically  
>> means UTF-8 locales if using translations. I was sure this was even  
>> mentioned in the prerequisites section of INSTALL, or at least in  
>> TRANSLATIONS, but obviously not.

Jan.

-- 
Do you need professional PHP or Horde consulting?
http://horde.org/consulting/