[Tickets #11930] Re: Horde_String::validUtf8 fails to validate valid UTF8

[noreply at bugs.horde.org]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/11930
------------------------------------------------------------------------------
  Ticket             | 11930
  Updated By         | Michael Rubinsky <mrubinsk at horde.org>
  Summary            | Horde_String::validUtf8 fails to validate valid UTF8
  Queue              | Horde Framework Packages
  Version            | Git master
  Type               | Bug
  State              | Resolved
  Priority           | 1. Low
  Milestone          |
  Patch              |
  Owners             | Michael Slusarz
------------------------------------------------------------------------------


Michael Rubinsky <mrubinsk at horde.org> (2013-01-09 22:40) wrote:

> From what i have seen  if this function returns false  the code  
> strips all non 7 bit characters
> This works in english but strips the whole text in other  
> languages.Maybe a less heavy handed approach is possible striping  
> only offending characters and replacing with equal byte symbols by  
> calling this logic .

This is only true for ActiveSync. In ActiveSync, the stripping of non  
7 bit characters is a last ditch effort when we can't determine what  
the encoding is. If we don't know what encoding the text is in, how  
are we to know what equal byte symbols are?

The reason we have to strip the non 7 bit characters is because if we  
send invalid UTF-8 data over wbxml, it can completely break the sync  
and even crash clients like iOS. Again, we only due this as a last  
ditch effort when the incoming email contains inproper character  
encoding information.