[imp] Odd symbols instead of mail message

[bvr]

We have this same problem, so I'll give you what information I have.

Also see this bug report: http://bugs.horde.org/ticket/10169

What you are seeing is a little-endian UTF-16 byte order mark displayed in latin1 character set.

http://en.wikipedia.org/wiki/Byte_order_mark

For us this problem occurs only on our debian (current stable) servers but doesn't on my fedora 16 desktop using the exact same code and IMAP server.

All of the messages that exhibit this problem are HTML formatted with a presumably wrong or invalid character set specification, like these:

<META http-equiv="Content-Type" content="text/html; charset="unicode">

But I've also seen this happen with utf-8 specified in the meta tag.

Notably all of these seem to be sent/processed by Microsoft software, Hotmail messages, but also Exchange and Outlook.

It could be that they are in fact UTF-16LE formatted but fail to specify it properly, the weird thing is that it doesn't happen on all server setups.

Some debugging points towards the XSS filtering that is done, most likely something with php's DOMDocument.

The servers that have this problem are running libxml 2.7.8.dfsg-2+squeeze1, but I've also compiled PHP against the libxml2-2.7.8 source with same results.

My desktop that doesn't have the problem is running libxml 2.7.8-6.fc16.

I've also tried copying /usr/lib/locale/locale-archive either way but I was unable to reproduce/fix the problem this way.

bvr.


On 01/21/2012 12:21 AM, Mat Cantin wrote:
> Greetings,
>
> Recently, some messages that are being displayed in Horde come up blank with the following odd symbols: ÿþ
>
> If for some reason you can't see it, it's a "y" with two dots and "bp" character.
>
> Seems to me like it's a character encoding problem with php, but I'm really not sure.
>
> This started happening recently only occasionally from one person, but now I'm starting to see it more often from multiple people. The people sending the messages are all using Outlook 2003.
>
> This only happens with IMP. I have Roundcube on the same server and it doesn't have any problems displaying the messages.
>
> Any ideas?
>