[Zope-dev] Re: request.get vs request.form.get

[Philipp von Weitershausen]

Maurits van Rees wrote:
> I have been busy with zope.formlib.  See this weblog entry for some
> background:
> 
> http://maurits.vanrees.org/weblog/archive/2007/11/search-forms-with-zope-formlib-and-batching
> 
> This is Plone 3 with Zope 2.10.4 btw.
> 
> 
> I discovered there is a difference between getting a value from the
> request or from request.form when unicode is involved.  From a pdb
> session:
> 
>>>> request.form.get('SearchableText')
> u'\xff'
>>>> request.get('SearchableText')
> '\xc3\xbf'
> 
> This is the character ÿ: a y with something extra on top. :) You can
> see the same in a python prompt (I hope this is readable for
> everyone):
> 
>>>> u"ÿ"
> u'\xff'
>>>> "ÿ"
> '\xc3\xbf'
> 
> 
> This gave me problems while using zope.formlib.  In the method
> zope.app.form.browser.textwidgets.TextWidget._getFormInput() the value
> is fetched from the request and not request.form. So you get a
> UnicodeDecodeError. In the weblog entry I mentioned I post some code
> to guard against that in a setUpWidgets method in your own form page:
> deliberately overwrite the variable in the request by the variable in
> request.form.
> 
> But what I want to ask here is: does something need to be fixed
> somewhere in zope?
> 
> Should the value in request and request.form really be the same
> always?  Probably: always unicode?

Out of the box, Zope 2 won't deal with unicode in the request at all.
The values in both the request and request.form are both 8bit encoded 
strings.

Now, the Zope3 form framework (incl. widgets) works with unicode only. 
So when we wanted to make Zope3-style forms (first zope.app.form, then 
zope.formlib) work in Zope 2 via Five, we either had to rewrite the 
whole widget story, or introduce a little hack. We went with the little 
hack, which takes request.form and decodes all values in it using the 
appropriate encoding that's guessed from the request. This is done in 
the same way that Zope 3 guesses the encoding.

That's why, when using Five's forms and *only then*, request.form will 
contain unicode.

I don't think we can move to unicode for the whole request (like Zope 3 
does) at all times since that will likely break backward compatibility. 
For instance, Archetypes is completely unaware of unicode IIRC...

> Should zope.app.form.browser.textwidgets.TextWidget._getFormInput()
> get the value from the form instead of from the request?

IMHO yes.