[kronolith] UTF8 problems with Lightning (was: Re: Kronolith with iCal r/w or task support or ... or ...)

[Lukas Haase]

Jan Schneider schrieb:
> Zitat von Lukas Haase <lukashaase at gmx.at>:
>> Jan Schneider schrieb:
>>> Zitat von Lukas Haase <lukashaase at gmx.at>:
>>>> Jan Schneider schrieb:
>>>>> Zitat von Lukas Haase <lukashaase at gmx.at>
>>>>>> Hi again,
>>>>>> Jan Schneider schrieb:
>>>>>>> Zitat von Lukas Haase <lukashaase at gmx.at>:
>>>>>>>> [...]
>>>>>>>> However apart from this I found problems with UTF8: When I 
>>>>>>>> create entried for example with Japanese characters in 
>>>>>>>> Thunderbird the items are just called ????? in Kronolith.
>>>>>>>>
>>>>>>>> The other way round, when I create an appointment with UTF8 
>>>>>>>> named in Kronolith they get somehow weird escaped. The wonderful 
>>>>>>>> Japanese christmas becomes: 
>>>>>>>> &#12463;&#12522;&#12473;&#12510;&#12473;
>>>>>>>>
>>>>>>>> Maybe it is enough to just add utf8_encode/decode?
>>>>>>>
>>>>>>> Did you configure the a compatible charset, e.g. UTF-8 for the 
>>>>>>> database backend?
>>>>>>
>>>>>> At first it was really at ISO8859...
>>>>>>
>>>>>> But now I set to "UTF-8" with the effect that all words containing 
>>>>>> umlauts have "?" now. But I think this is not such a big problem 
>>>>>> as there are not yet so many entries.
>>>>>
>>>>> And it's the expected behavior since your data in the database was 
>>>>> stored with ISO-8859-1.
>>>>>
>>>>>> Unfortunately the problem persists...same as described above.
>>>>>>
>>>>>> So it is actually implemented and should work? This would be very 
>>>>>> because then it would mean to just find the error in the config ;-)
>>>>>
>>>>> Yes, it should work. Does it work if you manually import an 
>>>>> iCalendar file exported from Thunderbird?
>>>>
>>>> Thank you very much.
>>>>
>>>> That works, surprisingly...
>>>>
>>>> But now something other very interesting occurs: When I access the 
>>>> calender again with Lightning the appointment is shown as "??" in 
>>>> Thunderbird ;-)
>>>>
>>>> Conclusion: Unicode on Clientside gehts ??? on Serverside and 
>>>> Unicode on Serverside gehts ??? on Clientside.
>>>>
>>>> So obviously there is a problem on the way with the WebDAV interface.
>>>
>>> Yes, looks like it. I guess the only way to further debug this is to 
>>> listen to the HTTP traffic between Thunderbird and Horde.
>>
>> Hmm, maybe you could tell me where which format needs to arrive?
>>
>> I have found now a really strange thing: When I just download the 
>> subscription URL with firofox everything is OK.
>>
>> But when I instead download with Total Commander or curl, I get ??? in 
>> the SUMMARY lines.
>>
>> Maybe I need to force Horde Somehow to send UTF8?
> 
> There is no need to *force* it. User agents have to tell Horde that they 
> are capable of UTF-8 by sending an Accept-Charset header. Or by being 
> Internet Explorer, that's too silly to send that header.

Hello,

Further testing: I think the Accept-Charset is not causing problems.

I added the following code on the top of rpc.php:

ob_start();
var_dump($_SERVER);
$dd = ob_get_contents();
ob_end_clean();
$f = fopen('/tmp/rpc.log', 'a');
fwrite($f, $dd);
fwrite($f, "======================================\n");
fclose($f);

Then I did a request from Thunderbird and Firefox the result is 
essentially the same. Apart from unimportant differences (request time, 
client port, HTTP_COOKIE, HTTP_PRAGMA, HTTP_CACHE_CONTROL which was 
either set in TB or in FF) I got this diff:

--- rpc.log-FF  2010-01-04 10:05:53.000000000 +0300
+++ rpc.log-TB  2010-01-04 10:04:13.000000000 +0300
@@ -1,14 +1,14 @@
    ["HTTP_USER_AGENT"]=>
-  string(106) "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.5) 
Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)"
+  string(122) "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; 
rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnen
hy/0.7.5.0"
    ["HTTP_ACCEPT"]=>
-  string(63) 
"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
+  string(99) 
"text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"
    ["HTTP_ACCEPT_LANGUAGE"]=>
-  string(47) "de-at,de-de;q=0.8,de;q=0.6,en-us;q=0.4,en;q=0.2"
+  string(35) "de-de,de;q=0.8,en-us;q=0.5,en;q=0.3"

Any other ideas?

UTF8 works with Google Calender as well as with a normal iCalender file...

Best Regards,
Luke