[Zope-CMF] Charsets

[Daniel Nouri]

Is this thread by any chance related to this bug:
https://bugs.launchpad.net/zope2/+bug/160968

  The IUserPreferredCharsets implementation of Zope 3 found in
  zope.publisher.http.HTTPCharsets has the following condition in it
  to check if the HTTP_ACCEPT_CHARSET header is available:

      header_present = 'HTTP_ACCEPT_CHARSET' in self.request

  However, with Zope 2's request will return '' (the empty string) for
  any header that starts with 'HTTP_', see
  ZPublisher.HTTPRequest.HTTPRequest.get.

  Ultimately, this results in the HTTPCharsets.getPreferredCharsets to
  return ['iso-8859-1'], where it should really return 'UTF-8'.

  To understand this problem better, look at
  Products.Five.browser.decode.processInputs, which uses the
  negotiator to find out which charset to use to convert form
  variables. For browsers that do not send the 'HTTP_ACCEPT_CHARSET'
  header, this will result in wrongly encoded form values. To
  reproduce this, fill in Chinese characters to any Five formlib form
  with Internet Explorer 6.0. Since Firefox sends HTTP_ACCEPT_CHARSET,
  it's not a problem there.

-- 
http://danielnouri.org