[horde] IMP: Encoding problem in message view
dgkim at dgkim.net
dgkim at dgkim.net
Thu Mar 17 10:47:37 UTC 2016
Deokgon Kim <dgkim at dgkim.net> wrote:
> 2016년 01월 12일 05:45에 Michael J Rubinsky 이(가) 쓴 글:
>>
>> Quoting 김 덕곤 <dgkim at dgkim.net>:
>>
>>> 2015. 12. 31. 오후 10:29에 Jan Schneider 이(가) 쓴 글:
>>>>
>>>> Zitat von 김 덕곤 <dgkim at dgkim.net>:
>>>>
>>>>> Hello.
>>>>>
>>>>> When I opening a message(that is not encoded in UTF-8),
>>>>> character is not shown in correct encoding.
>>>>>
>>>>> But, when I reply or forward it, correct character is shown.
>>>>> (And, the replied message is encoded in UTF-8)
>>>>>
>>>>> When I open a message as popup(Show all parts, and click on a part),
>>>>> character is also in a wrong encoding,
>>>>> but if I change character encoding to Unicode in a Browser option,
>>>>> correct character is shown.
>>>>> This means server already converted form encoding to encoding,
>>>>> but wrong Content-Type is sent.
>>>>>
>>>>> A message's encoding is 'ks_c_5601-1987'
>>>>> Web page's encoding is 'UTF-8'.
>>>>>
>>>>> Any suggestions?
>>>>>
>>>>> Versions.
>>>>> Horde : 5.2.8
>>>>> IMP : 6.2.11
>>>>>
>>>>> --
>>>>> Horde mailing list
>>>>> Frequently Asked Questions: http://horde.org/faq/
>>>>> To unsubscribe, mail: horde-unsubscribe at lists.horde.org
>>>>
>>>> https://bugs.horde.org/ticket/14149
>>>>
>>> I think I figured out why this happens.
>>>
>>> In Domhtml, at the line of 87, there is call to "convertCharset".
>>> as follows.
>>>
>>> 86 $this->_loadHTML(
>>> 87 $this->_xmlencoding .
>>> Horde_String::convertCharset($text, $c harset, 'UTF-8')
>>> 88 );
>>>
>>> Here is the call-stack dumped from Horde_Domhtml::__construct().
>>>
>>> __construct: Called from /usr/share/php/Horde/Text/Filter/Xss.php:72
>>> __construct 1. Horde_Text_Filter_Xss->postProcess()
>>> /usr/share/php/Horde/Text/Filter.php:108
>>> __construct 2. Horde_Text_Filter->filter()
>>> /usr/share/php/Horde/Core/Factory/TextFilter.php:72
>>> __construct 3. Horde_Core_Factory_TextFilter->filter() -:-
>>> __construct 4. call_user_func()
>>> /usr/share/php/Horde/Mime/Viewer/Base.php:438
>>> __construct 5. Horde_Mime_Viewer_Base->_textFilter()
>>> /usr/share/php/Horde/Mime/Viewer/Html.php:164
>>> __construct 6. Horde_Mime_Viewer_Html->_cleanHTML()
>>> /var/www/html/horde/imp/lib/Mime/Viewer/Html.php:179
>>> __construct 7. IMP_Mime_Viewer_Html->_IMPrender()
>>> /var/www/html/horde/imp/lib/Mime/Viewer/Html.php:75
>>> __construct 8. IMP_Mime_Viewer_Html->_renderInline()
>>> /usr/share/php/Horde/Mime/Viewer/Base.php:156
>>> __construct 9. Horde_Mime_Viewer_Base->render()
>>> /var/www/html/horde/imp/lib/Contents.php:654
>>> __construct 10. IMP_Contents->renderMIMEPart()
>>> /var/www/html/horde/imp/lib/Mime/Viewer/Alternative.php:128
>>> __construct 11. IMP_Mime_Viewer_Alternative->_IMPrender()
>>> /var/www/html/horde/imp/lib/Mime/Viewer/Alternative.php:54
>>> __construct 12. IMP_Mime_Viewer_Alternative->_renderInline()
>>> /usr/share/php/Horde/Mime/Viewer/Base.php:156
>>> __construct 13. Horde_Mime_Viewer_Base->render()
>>> /var/www/html/horde/imp/lib/Contents.php:654
>>> __construct 14. IMP_Contents->renderMIMEPart()
>>> /var/www/html/horde/imp/lib/Contents.php:1462
>>> __construct 15. IMP_Contents->getInlineOutput()
>>> /var/www/html/horde/imp/lib/Ajax/Application/ShowMessage.php:296
>>> __construct 16. IMP_Ajax_Application_ShowMessage->showMessage()
>>> /var/www/html/horde/imp/lib/Dynamic/Message.php:64
>>> __construct 17. IMP_Dynamic_Message->_init()
>>> /var/www/html/horde/imp/lib/Dynamic/Base.php:90
>>> __construct 18. IMP_Dynamic_Base->__construct()
>>> /var/www/html/horde/imp/dynamic.php:33
>>>
>>> I tried to find where the 'convertCharset' occurs.
>>>
>>> I found in 'Horde/Mime/Viewer/Html.php'. (line 106 to 114)
>>>
>>> 106 protected function _renderInline()
>>> 107 {
>>> 108 $html =
>>> $this->_cleanHTML($this->_mimepart->getContents(), array('in
>>> line' => true));
>>> 109
>>> 110 return $this->_renderReturn(
>>> 111 Horde_String::convertCharset($html->returnHtml(),
>>> $this->_mimepa rt->getCharset(), 'UTF-8'),
>>> 112 'text/html; charset=UTF-8'
>>> 113 );
>>> 114 }
>>>
>>> So I think, 'convertCharset' is unintentionally called twice.
>>
>> Horde_Mime_Viewer::_renderInline() isn't (shouldn't) be called in
>> this context, since we are in IMP_Mime_Viewer_Html, which overrides
>> the parent's copy of the _renderInLine() method.
>>
>>
>>
>
> Hello.
> Today I tried to figure out a little more.
>
> I do understand Horde_Mime_Viewer::_renderinline() isn't called at
> this context.
>
> And I still have the problem.
>
> When I change code not to 'convertCharset', It works.
> (/usr/share/php/Horde/Domhtml.php)
>
> 83 /* Convert/try with UTF-8 first. */
> 84 $this->_origCharset = Horde_String::lower($charset);
> 85 $this->_xmlencoding = '<?xml encoding="UTF-8"?>';
> 86 $this->_loadHTML(
> 87 $this->_xmlencoding .
> Horde_String::convertCharset($text, $charset, 'UTF-8')
> 88 );
>
> At the line of 87.
>
> Why, in the first place, converting to UTF-8?
>
> And
> what if body part of html($text) doesn't contains fully structured HTML,
> Can Domhtml.php handle it?
> (I tested DOMDocument object, if the $text doesn't contains [meta
> charset], 'dom->encoding' is not working.)
>
> And what is the state of $text?
> Does Content-Transfer-Encoding is processed?
>
> To confirm, this problem is all related to Domhtml.php,
> what can be the status of member variables?
> Or is this problem related to php DOMDocument?
> --
> Horde mailing list
> Frequently Asked Questions: http://horde.org/faq/
> To unsubscribe, mail: horde-unsubscribe at lists.horde.org
Hello.
Today, I struggled for an hour.
I guess, the problem is caused by library like iconv.
After several error_log() stuff. I found
/usr/share/php/Horde/Text/Filter/Cleanhtml.php is not working.
CODE
public function postProcess($text)
{
(d)error_log("cleanhtml before text : " . $text);
(d)error_log("cleanhtml charset : " . $this->_params['charset']);
if (!Horde_Util::extensionExists('tidy') ||
(($this->_params['size'] !== false) &&
(strlen($text) > $this->_params['size']))) {
return $text;
}
$tidy_config = array(
'enclose-block-text' => true,
'hide-comments' => true,
'indent' => false,
'numeric-entities' => true,
'preserve-entities' => true,
'show-body-only' => !empty($this->_params['body_only']),
'tab-size' => 0,
'wrap' => 0
);
$tidy = new tidy();
if (Horde_String::lower($this->_params['charset']) == 'us-ascii') {
if ($tidy->parseString($text, $tidy_config, 'ascii')) {
$tidy->cleanRepair();
$text = $tidy->value;
}
(1) } elseif
($tidy->parseString(Horde_String::convertCharset($text,
$this->_params['charset'], 'UTF-8'), $tidy_config, 'utf8')) {
$tidy->cleanRepair();
(d)error_log("cleanhtml tidy worked : " . $tidy->value);
(3)if ( $this->_params['charset'] == 'ks_c_5601-1987' ) {
(3)$this->_params['charset'] = 'EUC-KR';
(3)}
(2) $text = Horde_String::convertCharset($tidy->value,
'UTF-8', $this->_params['charset']);
}
(d)error_log("cleanhtml after text : " . $text);
return $text;
}
/CODE
(1) convertCharset is working. I guess.
(2) convertCharset is not working.
So I changed 'ks_c_5601-1987' to 'EUC-KR'. (3)
It is working now.
Is this right guess?
More information about the horde
mailing list