[Zope-dev] unicode+make_query - New Pull request

[Michael McFadden]

Tres:

I hope I got everything you wanted.  Thanks for taking the time to give feedback!   I learned a lot doing this.

https://github.com/zopefoundation/Zope/pull/5

In an effort to increase confidence that existing functionality is not affected, I did not modify any existing tests. I only created new tests.


Notes:
-----
I abandoned the idea to add the 'enc' parameter

Since make_query takes **kwargs - which are designed to be 'key'='value' pairs, no matter what variable name I choose, I have a chance of breaking
someone's install.
I could break our install, for example, by picking kwargs['encoding'], because we use 'encoding' as a key in a query string. It becomes ambiguous if the  argument 'encoding' is supposed to be used as a key in the query string or to control make_query() behavior.  It's way to risky to implement this way.

Thoughts?

-----
I'm still thinking about the current 'make_query' in the field.  
If you provide Unicode types to make_query, and they are being successfully encoded with ASCII, all is well - but the output will change.

Example:
 value=u'Hello'
 make_query(key=value)  #returns 'key=Hello'

New functionality would return:
 'key:utf8:ustring=Hello'

This might not be wanted.

What do you think about preserving backward-compatibility with something like:

if isInstance(value, unicode):
    try:
        string = str(value)
    except UnicodeDecodeError:
        string = value.encode(utf8)

Which would preserve existing behavior.

Thoughts?

-----
I went slightly overboard and re-factored complex_marshal to use a recursive solution for nested structures.
 I didn't want to type check for unicode and v.decode() every time simple_marshal() was called. 
  that's all it was.  My OCD took over.
 It looks cleaner to my eyes!

All existing tests pass, plus support for lists in lists, records in lists, lists in records, records in records.

It might be a good idea to limit depth to, say, 3.  Right now depth is limited to python recursion depth.

Thoughts?


-Flip