problems with accented characters - need advice
hello, we are looking for a simple solution to ensure that objects added to the zope database have valid names... including those which start out containing accented characters ( we are in europe ). we are considering making the change in our PUT, located in NullResource.py in the webdav area. we are wondering if anyone has any comments about this solution... is it a good idea. anyone have any other ideas? we have a process method for correcting accented characters and is located in the constructor of the object class : it works like ob.id = processID(name) here is the change...we have added 'ob.id or name' to setObject method request. <code starts here> # Delegate actual PUT handling to the new object. ob.PUT(REQUEST, RESPONSE) self.__parent__._setObject(ob.id or name, ob) Eric Brun Savoie Technologie Savoie Technolac
There is a function defined in ObjectManager.py called checkValidId. You could override this to enforce certain rules either in a subclass of OM (overridding _setObject) or by a direct monkey patch of ObjectManager.py. Then it would not be possible to supply invalid names from anywhere (ZMI, DAV, FTP or programmatically). hth, -Casey On Tue, 2002-06-18 at 13:08, Eric Brun wrote:
hello,
we are looking for a simple solution to ensure that objects added to the zope database have valid names... including those which start out containing accented characters ( we are in europe ).
we are considering making the change in our PUT, located in NullResource.py in the webdav area. we are wondering if anyone has any comments about this solution... is it a good idea. anyone have any other ideas?
we have a process method for correcting accented characters and is located in the constructor of the object class : it works like ob.id = processID(name)
here is the change...we have added 'ob.id or name' to setObject method request.
<code starts here> # Delegate actual PUT handling to the new object. ob.PUT(REQUEST, RESPONSE) self.__parent__._setObject(ob.id or name, ob)
Eric Brun Savoie Technologie Savoie Technolac
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Eric Brun writes:
we are looking for a simple solution to ensure that objects added to the zope database have valid names... including those which start out containing accented characters ( we are in europe ). I would get rid of the id restriction...
In fact, there is no real reason that Zope ids must be valid URL path elements. The reason why it is still there is that a change should work for all languages and not only western ones. This poses the question how the byte string representing the id your should be URL quoted. A good solution must probably wait until Zope id's are Unicode strings. Dieter
On Tuesday 18 Jun 2002 8:44 pm, Dieter Maurer wrote:
The reason why it is still there is that a change should work for all languages and not only western ones. This poses the question how the byte string representing the id your should be URL quoted.
There is an RFC, I forget which one, which specifies utf8.
Toby Dickenson writes:
The reason why it is still there is that a change should work for all languages and not only western ones. This poses the question how the byte string representing the id your should be URL quoted.
There is an RFC, I forget which one, which specifies utf8. I have seen it in the HTML 4 specification. But, I am unsure whether it is already implemented by browsers.
It would answer the question about the destination encoding. We must also define the source encoding (i.e. that of the byte string representing the id). This could be UTF-8, Python's default encoding or determined by an environment variable. Dieter
Toby Dickenson <tdickenson@geminidataloggers.com> wrote:
On Tuesday 18 Jun 2002 8:44 pm, Dieter Maurer wrote:
The reason why it is still there is that a change should work for all languages and not only western ones. This poses the question how the byte string representing the id your should be URL quoted.
There is an RFC, I forget which one, which specifies utf8.
I'm not sure about that, last time I digged I found that it was explicitely said in the RFCs that no encoding was specified and that it was up to the application to decide what to use. Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com
Florent Guillaume wrote:
Toby Dickenson <tdickenson@geminidataloggers.com> wrote:
On Tuesday 18 Jun 2002 8:44 pm, Dieter Maurer wrote:
The reason why it is still there is that a change should work for all languages and not only western ones. This poses the question how the byte string representing the id your should be URL quoted.
There is an RFC, I forget which one, which specifies utf8.
I'm not sure about that, last time I digged I found that it was explicitely said in the RFCs that no encoding was specified and that it was up to the application to decide what to use.
Florent
I came across a nice site regarding UTF-8 and unicode, perhaps this helps: http://www.cl.cam.ac.uk/~mgk25/unicode.html cheers, oliver
Florent Guillaume <fg@nuxeo.com> wrote:
Toby Dickenson <tdickenson@geminidataloggers.com> wrote:
On Tuesday 18 Jun 2002 8:44 pm, Dieter Maurer wrote:
The reason why it is still there is that a change should work for all languages and not only western ones. This poses the question how the byte string representing the id your should be URL quoted.
There is an RFC, I forget which one, which specifies utf8.
I'm not sure about that, last time I digged I found that it was explicitely said in the RFCs that no encoding was specified and that it was up to the application to decide what to use.
To be precise: RFC 2068 (HTTP 1.1) says: For definitive information on URL syntax and semantics, see RFC 1738 [4] and RFC 1808 [11]. The BNF above includes national characters not allowed in valid URLs as specified by RFC 1738, since HTTP servers are not restricted in the set of unreserved characters allowed to represent the rel_path part of addresses, and HTTP proxies may receive requests for URIs not defined by RFC 1738. RFC 1808 (URL) doesn't talk about charset or bytes. RFC 2396 (URI, updates RFC 1808 & RFC 1738) says, and I'll quote extensively. Note the last two paragraphs. 2.1 URI and non-ASCII characters The relationship between URI and characters has been a source of confusion for characters that are not part of US-ASCII. To describe the relationship, it is useful to distinguish between a "character" (as a distinguishable semantic entity) and an "octet" (an 8-bit byte). There are two mappings, one from URI characters to octets, and a second from octets to original characters: URI character sequence->octet sequence->original character sequence A URI is represented as a sequence of characters, not as a sequence of octets. That is because URI might be "transported" by means that are not through a computer network, e.g., printed on paper, read over the radio, etc. A URI scheme may define a mapping from URI characters to octets; whether this is done depends on the scheme. Commonly, within a delimited component of a URI, a sequence of characters may be used to represent a sequence of octets. For example, the character "a" represents the octet 97 (decimal), while the character sequence "%", "0", "a" represents the octet 10 (decimal). There is a second translation for some resources: the sequence of octets defined by a component of the URI is subsequently used to represent a sequence of characters. A 'charset' defines this mapping. There are many charsets in use in Internet protocols. For example, UTF-8 [UTF-8] defines a mapping from sequences of octets to sequences of characters in the repertoire of ISO 10646. In the simplest case, the original character sequence contains only characters that are defined in US-ASCII, and the two levels of mapping are simple and easily invertible: each 'original character' is represented as the octet for the US-ASCII code for it, which is, in turn, represented as either the US-ASCII character, or else the "%" escape sequence for that octet. For original character sequences that contain non-ASCII characters, however, the situation is more difficult. Internet protocols that transmit octet sequences intended to represent character sequences are expected to provide some way of identifying the charset used, if there might be more than one [RFC2277]. However, there is currently no provision within the generic URI syntax to accomplish this identification. An individual URI scheme may require a single charset, define a default charset, or provide a way to indicate the charset used. It is expected that a systematic treatment of character encoding within URI will be developed as a future modification of this specification. Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com
participants (6)
-
Casey Duncan -
Dieter Maurer -
Eric Brun -
Florent Guillaume -
Oliver Bleutgen -
Toby Dickenson