Maurits van Rees wrote:
I have been busy with zope.formlib. See this weblog entry for some background:
http://maurits.vanrees.org/weblog/archive/2007/11/search-forms-with-zope-for...
This is Plone 3 with Zope 2.10.4 btw.
I discovered there is a difference between getting a value from the request or from request.form when unicode is involved. From a pdb session:
request.form.get('SearchableText') u'\xff' request.get('SearchableText') '\xc3\xbf'
This is the character ÿ: a y with something extra on top. :) You can see the same in a python prompt (I hope this is readable for everyone):
u"ÿ" u'\xff' "ÿ" '\xc3\xbf'
This gave me problems while using zope.formlib. In the method zope.app.form.browser.textwidgets.TextWidget._getFormInput() the value is fetched from the request and not request.form. So you get a UnicodeDecodeError. In the weblog entry I mentioned I post some code to guard against that in a setUpWidgets method in your own form page: deliberately overwrite the variable in the request by the variable in request.form.
But what I want to ask here is: does something need to be fixed somewhere in zope?
Should the value in request and request.form really be the same always? Probably: always unicode?
Out of the box, Zope 2 won't deal with unicode in the request at all. The values in both the request and request.form are both 8bit encoded strings. Now, the Zope3 form framework (incl. widgets) works with unicode only. So when we wanted to make Zope3-style forms (first zope.app.form, then zope.formlib) work in Zope 2 via Five, we either had to rewrite the whole widget story, or introduce a little hack. We went with the little hack, which takes request.form and decodes all values in it using the appropriate encoding that's guessed from the request. This is done in the same way that Zope 3 guesses the encoding. That's why, when using Five's forms and *only then*, request.form will contain unicode. I don't think we can move to unicode for the whole request (like Zope 3 does) at all times since that will likely break backward compatibility. For instance, Archetypes is completely unaware of unicode IIRC...
Should zope.app.form.browser.textwidgets.TextWidget._getFormInput() get the value from the form instead of from the request?
IMHO yes.