[Zope-dev] IUnicodeEncodingConflictResolver moronosity

[Hanno Schlichting]

On Tue, Sep 21, 2010 at 8:17 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> I like the idea of IUnicodeEncodingConflictResolver, ...

[...]

> Why has no-one noticed this?

Because there's not all that many people left using plain Zope 2 and
upgrading to new versions.

The rest is using Plone, which has its own version of this:

def _unicode_replace(structure):
    if isinstance(structure, str):
        text = structure.decode('utf-8', 'replace')
    else:
        text = unicode(structure)
    return text

class UTF8EncodingConflictResolver(object):
    implements(IUnicodeEncodingConflictResolver)

    def resolve(self, context, text, expression):
        return _unicode_replace(text)

We can do this, as we force manage_page_charset and
default-zpublisher-encoding to be utf-8 and only support utf-8 and
Unicode in the database.

Zope 2 itself tries to support arbitrary encodings, at which point it
can only guess badly or fail.

Hanno