- Zope-Dev - Zope lists

Fwd: [ZPT] Making PageTemplate's edit pages Unicode aware
by Stuart Bishop 09 Apr '04

09 Apr '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I havn't received much feedback on the ZPT mailing list, so I thought I'd bring it over here to a wider audience (thread is at http://mail.zope.org/pipermail/zpt/2004-March/005218.html ). Begin forwarded message: > From: Stuart Bishop <stuart.b(a)commonground.com.au> > Date: 29 March 2004 6:13:06 PM > To: Dieter Maurer <dieter(a)handshake.de> > Cc: zpt(a)zope.org > Subject: Re: [ZPT] Makeing PageTemplate's edit pages Unicode aware > > On 27/03/2004, at 9:57 PM, Dieter Maurer wrote: > >> Stuart Bishop wrote at 2004-3-25 12:27 +1100: >>> Currently, if you enter non-ascii text into the title or contents >>> fields on a PageTemplate's edit page, the data ends up stored as >>> an encoded string (using management_page_charset, if it is set. >>> Unknown >>> encoding if it is not). >>> >>> This should be easy to fix using the foo:charset:ustring notation >>> to have Zope convert the encoded strings to Unicode. However, the >>> file upload feature is more problematic. Should the file upload >>> try converting the file to Unicode from UTF-8 and raise an exception >>> if this is not possible? I personally feel this is preferable to >>> ending up with arbitrarily enncoded document source, with no idea >>> of the character set used. >> >> I do not think that Zope should convert when it does not know the >> encoding. I am unaware that a missing "management_page_charset" >> can be interpreted as "UTF-8". If this were the case, converstion >> to unicode might be correct. By the way: the HTML specification >> says that uploaded files should come with a "content-type" >> declaration. >> In this case, the charset specified there (if any) should be used >> to determine the encoding. > > Yes - A missing management_page_charset should probably be interpreted > as either US-ASCII or ISO-8859-1. US-ASCII is probably more correct, > but I would guess that most browsers will be configured to use > ISO-8859-1 as their default (and this might be specified in the HTML > spec?) > > I guess using the charset type the browser tells us for file uploads > means we can blame the browser. I don't know how this could be reliable > (since text files themselves don't encode their character set unless > they happen to be UTF-16 or have a BOM). I am wondering if having a > file upload function is incompatible with a Unicode aware page > templates product. > > If management_page_charset is not set, it is unknown what charset > is being used. The only way of knowing the character set of data that > has been submitted is to know the character set of the form that it > was submitted from. All other mechanisms do not work due to > incompatibilities in how the browsers work. > > Currently, if you create a page template that contains non-ASCII > characters, any tal:content or tal:replace expressions that return > Unicode will now raise a Unicode error. This can be demonstrated > simply: > <html> > <div>My 2¢</div> > <div tal:content="python:u'My 2\N{CENT SIGN}'">My 2¢</div> > </html> > > These are the things I think need to be fixed in Zope's Page Templates > implementation to make them Unicode aware. There may be more (?): > > - It should be possible for the actual page template source to > be stored as a Unicode string. Currently, there is an assert > ensuring it is a traditional string. > > - The title property should be a Unicode string. > > - PageTemplateFile should grow an optional charset parameter, > defaulting to US-ASCII. > > - PageTemplate.write(text) should raise an exception if text > is not either a Unicode string or an ASCII string. > > - The ZopePageTemplate edit page should use Zope's > :charset:ustring notation so Unicode strings get passed > to its handler. > > - The file upload widget needs to either be removed, or grow > a charset box. I don't think either of these solutions are > ideal :-( > > Note that when I say 'Unicode string', we can still store ASCII > text using a traditional string to save space. > > My application is currently using a ZopePageTemplate subclass that > has been modified to use Unicode strings for the document source > and title, and it seems to be functioning just fine. Does anyone > know if that "assert type(text) == type('')" in PageTemplate.write > is there for a reason? - -- Stuart Bishop <stuart(a)stuartbishop.net> http://www.stuartbishop.net/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAdfPfAfqZj7rGN0oRAkBuAJ0WLSC3V2eL+zNzkQqBqjJ2bl5degCfe2SB DlT7NTsieQlDhVgEnHYaXp8= =6XPE -----END PGP SIGNATURE-----

1 0

Using a truely revision based storage for Zope ?
by Jean-Francois.Doyon＠CCRS.NRCan.gc.ca 08 Apr '04

08 Apr '04

Hello, A co-worker of mine had an idea I thought was interesting and would like to investigate further. First of all, the history behind: We had requirements for: - Out of synch editing: Editing objects that are currently "published" and publishing those only once they've been workflowed (Obviously we're using the CMF here). This is currently achieved using multiple servers, one with a pubished version of the content, and another where the workflow and editing happens. Pretty standard stuff. - Out of synch publishing. certain pieces of content can be published, regardless of the state of the parent or vice versa. Something like I edit a parent, I edit a child, I want to edit a child despite the parent being in a unpublishable state, and so on. this gets conciderably more complicated. Said co-worker went through alot of pain to do manage multiple content trees, do tree diff'ing and so on. - Archiving of all these versions of content. - Handling of multilingual content. - more that are too specific to our need to mention here, basically, highly complex workflows and content types. So something has been developped that meets these requirements, and it's complicated as hell. The development side wraps content objects to make them multilingual, there are 3 content trees maintained. In some cases the content types differ on both sites due to workflow requirements and the fact that the publshing side was already being developped. (Developement was disjointed, an unfortunate situation, but couldn't be avoided). So out of all this, I'm really wanting a better way to do this. Althogh it works, it's rather complicated to maintain, and some of it requires changing the way certain things are done when developping content. So the basic theory is: Wouldn't it be great if we could just have a storage system for Zope that is truly revisioned (I will use this term instead of "versions" as not to get mixed up with Zope versions) ??? i.e. A system where a given object would exist twice somehow ... The revision for publishing, and the other revision going through workflow/editing/etc ... This alone would solve many of the requirements above without the need of many servers and publishing/development/pending tree magic. So my first big question is this: Do Zope internals provide enough flexibility to support such a storage system ? A storage system that actually stores, in a basically permanent fashion, the various revisions of an object ? (I don't mean just historie's/undo's, i mean revision controlled objects in the SCM type sense). Can Zope be made to publish an object using one revision of it, while editing another ? Without re-writing the whole thing of course :) Here's what said co-worker (I keep saying that because he deserves credit for what I think is a great idea) came up with: Using Subversion, and it's Python API, as the storage system for Zope! Perceived benefits: - Revisionned objects: Just remember what revision of the object you want published, and use that ... Editing/Workflow and so on can happen on further revisions. At some point the revision to be published gets changed, and voila. - Efficient storage due to the fact it stores only the differences ... - Call back hooks are available in the python API, so there can be notification of changes in the storage, which introduces interesting possiblities for shared storage, ZEO style. I don't know how ZEO works internally, but I know there's the notification system to expire content in the caches of the clients. Does ZODB do the "notification" or is it in the ZEO code layer ? In this case, the storage itself could provide notification. - Full archival of the objects over time, efficiently. - Benefits from subversion's storage with is BDB 4.2 right now ... - Possibility of benefitting from SVN's "cheap copy" mechanism (sort of OO based, references are used until the content changes). - Many others I'm sure ... I've started looking at the ZODB and APE packages to try and get some understanding of how the whole storage interaction works, but it'll take me some time to figure it all out ... So I thought I'd get feedback on the idea first ... Thoughts, comments ? Thanks, Jean-François Doyon Internet Service Development and Systems Support / Développement des services et soutien de systèmes Internet GeoAccess Division / Division GéoAccès Canada Center for Remote Sensing / Centre canadien de télédétection Natural Resources Canada / Ressources naturelles Canada http://atlas.gc.ca Phone: (613) 992-4902 Fax: (613) 947-2410

2 1

manage_permission from script
by Erik A.Dahl 08 Apr '04

08 Apr '04

I'm trying to use manage_permission from a script against the root application object. When I do I get the following error from ObjectManager. This definitely works from within zope since I'm able to edit permissions from the ZMI. I guess the app object returned from Zope.app() doesn't have an acquisition wrapper? -EAD >>> app.manage_permission("View", ['Owner',], 1) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/home/edahl/zope-2.7.0/lib/python/AccessControl/Role.py", line 164, in manage_permission for p in self.ac_inherited_permissions(1): File "/home/edahl/zope-2.7.0/lib/python/AccessControl/Role.py", line 82, in ac_inherited_permissions for p in self._subobject_permissions(): File "/home/edahl/zope-2.7.0/lib/python/OFS/ObjectManager.py", line 212, in _subobject_permissions return (Products.__ac_permissions__+ AttributeError: aq_acquire

3 4

Re: CVS: Zope3/src/zope/tal - talparser.py:1.6
by Philipp von Weitershausen 08 Apr '04

08 Apr '04

Gintautas Miliauskas wrote: > Update of /cvs-repository/Zope3/src/zope/tal > In directory cvs.zope.org:/tmp/cvs-serv27951/src/zope/tal > > Modified Files: > talparser.py > Log Message: > Removed an assertion which disallows usage of Zope3 i18n in XML markup. > > Added a few test cases which run the new code path. > > > === Zope3/src/zope/tal/talparser.py 1.5 => 1.6 === > --- Zope3/src/zope/tal/talparser.py:1.5 Fri Mar 19 16:42:04 2004 > +++ Zope3/src/zope/tal/talparser.py Wed Apr 7 11:13:00 2004 > @@ -81,7 +81,6 @@ > taldict[keybase] = value > item = item + ("tal",) > elif ns == 'i18n': > - assert 0, "dealing with i18n: " + `(keybase, value)` > i18ndict[keybase] = value > item = item + ('i18n',) > fixedattrlist.append(item) Hello, I would like to backport this patch (including tests) to Zope 2, since I need to i18n XML generated by ZPTs. Any objections? Philipp

2 1

setup.py and unittests
by yuppie 07 Apr '04

07 Apr '04

Hi! Trying to run unittests from a Zope HEAD installation, I stumbled over these setup.py issues: - the new 'transaction' package is missing - testrunner.py is installed, not test.py - some files needed to run unittests are missing - SearchIndex.UnIndex still imports BTree and IOBTree which were removed from setup.py 4 months ago Proposals: - install test.py instead of the deprecated testrunner.py - install all files needed to run the unittests - fix all test failures - would be nice if at least the nightly tests could be run from a clean new installation instead of a sandbox Cheers, Yuppie

1 0

OrderSupport: small improvement
by yuppie 07 Apr '04

07 Apr '04

Hi! Currently the OrderSupport API always orders sub-objects relative to the complete list of sub-objects. This causes problems if some hidden sub-objects are not shown. From an UI point of view, moving sub-objects should always be relative to the visible subset of sub-objects. I'd like to add a new optional argument to moveObjectsByDelta (and the derived methods moveObjectsUp, moveObjectsDown, moveObjectsToTop, moveObjectsToBottom): subset_ids This would be 100% backwards compatible and would allow to specify the visible subset of sub-objects. def moveObjectsByDelta(ids, delta, subset_ids=None): If subset_ids is specified, delta will be interpreted relative to that subset. If there are no objections I'll soon check in the necessary changes to Zope-2_7-branch and HEAD. Cheers, Yuppie

1 0

Next bug: base_edit and webdav lock
by Christian Heimes 06 Apr '04

06 Apr '04

The base_edit template of archetypes doesn't take care of webdav looks. It's possible to save a file while it's looked. Christian

1 1

Re: [Zope-dev] Site Error
by AP Meyer 06 Apr '04

06 Apr '04

Maybe I just don't see it, but with the latest Archetypes from the CVS I get the following error: Site error This site encountered an error trying to fulfill your request. The errors were: Error Type AttributeError Error Value reference_catalog Request made at 2004/04/06 16:23:41.266 GMT+2 What I found so far is that the reference_catalog problem only occurs in an instance of Project from the ArchGenXMLComplexExample code. Maybe this is a known bug? thanks Andre Dieter Maurer wrote: > AP Meyer wrote at 2004-4-2 11:09 +0200: > >>Even more disturbing, however, I frequently get the following error >>message and am not sure what it means and whether it is related to Plone >>2.0/Archetypes 1.2.5-rc2 or just Zope 2.7. >> >> >>Site error >> >>This site encountered an error trying to fulfill your request. The >>errors were: >> >>Error Type >> ValueError >>Error Value >> Cache values may only be in one cache. >>Request made at > > > This is a clear sign of a persistency bug: > > Persistent objects must not be stored outside of their connection, > neither in module global data structure nor as class variables. > > A few days ago, I fixed two persistency bugs (one major and one minor) > in Archetypes CVS HEAD. > > With these fixes, our AT based products are again refreshable (in fact, > the fix for the major bug was already sufficient for this; > occurence of the minor bug is unlikely). Let me know should you > still have problems with the fixes. > -- Dr. Andre P. Meyer http://home.hccnet.nl/a.meyer/ TNO FEL Command & Control and Simulation, http://www.fel.tno.nl/div2/ Delft Cooperation on Intelligent Systems, http://www.decis.nl/ -- ------------------------------------------------------------------------------ The disclaimer that applies to e-mail from TNO Physics and Electronics Laboratory can be found on: http://www.tno.nl/disclaimer/email.html ------------------------------------------------------------------------------

1 0

Unique ID generation with ZODB.
by Syver Enstad 06 Apr '04

06 Apr '04

How does one implement a unique id generator in ZODB. The id's should be strictyl ascending and be a an int. Would opening a temporary connection increment a persistent counter and commit do the trick?

3 3

External Editor and Archetypes 1.3
by Christian Heimes 05 Apr '04

05 Apr '04

Hi! I tested the External Editor integration with the last version of AT Document from my AT Content Types. When saving the changes I'm getting this exception: Traceback (innermost last): Module ZPublisher.Publish, line 100, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 40, in call_object Module Products.Archetypes.BaseContent, line 81, in PUT Module Products.Archetypes.Marshall, line 95, in demarshall Module Products.ATContentTypes.types.ATDocument, line 94, in setText Module Products.Archetypes.Field, line 701, in set Module Products.Archetypes.BaseUnit, line 23, in __init__ Module Products.Archetypes.BaseUnit, line 32, in update Module Products.PortalTransforms.MimeTypesRegistry, line 231, in __call__ Module Products.PortalTransforms.MimeTypesRegistry, line 181, in classify Module Products.PortalTransforms.zope.MimeTypesTool, line 65, in lookup Module Products.PortalTransforms.MimeTypesRegistry, line 126, in lookup - __traceback_info__: ("'None'", 'None') Module Products.PortalTransforms.MimeTypesRegistry, line 269, in split MimeTypeException: Malformed MIME type (None) Christian

1 0