March 2008 - Zope-Dev

ZCatalog: updateMetadata and comparing string and unicode
by Maurits van Rees 06 Mar '08

06 Mar '08

Hi all, I have an item in the portal_catalog of my Plone site that has some string as description. The real object meanwhile has had a code change so the description field now returns unicode. When I now recatalog that object it throws an error: Module Products.ZCatalog.Catalog, line 359, in catalogObject Module Products.ZCatalog.Catalog, line 318, in updateMetadata UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 159: ordinal not in range(128) > /home/maurits/buildout/projectdeploy/parts/zope2/lib/python/Products/ZCatalog/Catalog.py(318)updateMetadata() -> if data.get(index, 0) != newDataRecord: This happens when the current data in the catalog get compared to the new data. If there is a difference, the new data is stored. But to compare the old string with the new unicode the string is converted to unicode. This fails because the string has non ascii characters in it. So basically what happens is this error: >>> unicode("Ã¤", 'utf-8') == u"Ã¤" True >>> "Ã¤" == u"Ã¤" Traceback (most recent call last): File "<stdin>", line 1, in ? UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128) Logical enough. This can be fixed in ZCatalog: maurits@hobb:~/svn/Zope-210/lib/python/Products/ZCatalog $ svn diff Index: Catalog.py =================================================================== --- Catalog.py (revision 84388) +++ Catalog.py (working copy) @@ -304,7 +304,15 @@ # meta_data is stored as a tuple for efficiency data[index] = newDataRecord else: - if data.get(index, 0) != newDataRecord: + try: + changed = data.get(index, 0) != newDataRecord + except UnicodeDecodeError: + # Converting some string to unicode fails. This + # conversion happens when a string and a unicode need + # to be compared. Those two are not the same, so + # logically there has been a change, so: + changed = True + if changed: data[index] = newDataRecord return index Index: tests/testCatalog.py =================================================================== --- tests/testCatalog.py (revision 84388) +++ tests/testCatalog.py (working copy) @@ -1,3 +1,4 @@ +# -*- coding: utf-8 -*- ############################################################################## # # Copyright (c) 2002 Zope Corporation and Contributors. All Rights Reserved. @@ -177,6 +177,13 @@ def __nonzero__(self): self.fail("__nonzero__() was called") +class zdummyText(ExtensionClass.Base): + def __init__(self, text): + self.text = text + + def title(self): + return self.text + class FakeTraversalError(KeyError): """fake traversal exception for testing""" @@ -261,6 +268,12 @@ data = self._catalog.getMetadataForUID('1') self.assertEqual(data['title'], '1') + text = zdummyText('A string with an accent: \xc3\xa4.') + self._catalog.catalog_object(text, '1') + text.text = unicode("A simple unicode.") + self._catalog.catalog_object(text, '1') + + def testReindexIndexDoesntDoMetadata(self): self.d['0'].num = 9999 self._catalog.reindexIndex('title', {}) =================================================================== With that change it works: on the live site I can edit and save that item without errors. Without the change to the code, the added test fails at precisely the point where the change should be done. But if I change the code the test still fails because something similar goes wrong in the KeywordIndex, with this traceback: =================================================================== Error in test testUpdateMetadata (Products.ZCatalog.tests.testCatalog.TestZCatalog) Traceback (most recent call last): File "unittest.py", line 260, in run testMethod() File "/home/maurits/svn/Zope-210/lib/python/Products/ZCatalog/tests/testCatalog.py", line 274, in testUpdateMetadata self._catalog.catalog_object(text, '1') File "/home/maurits/svn/Zope-210/lib/python/Products/ZCatalog/ZCatalog.py", line 536, in catalog_object update_metadata=update_metadata) File "/home/maurits/svn/Zope-210/lib/python/Products/ZCatalog/Catalog.py", line 368, in catalogObject blah = x.index_object(index, object, threshold) File "/home/maurits/svn/Zope-210/lib/python/Products/PluginIndexes/common/UnIndex.py", line 235, in index_object res += self._index_object(documentId, obj, threshold, attr) File "/home/maurits/svn/Zope-210/lib/python/Products/PluginIndexes/KeywordIndex/KeywordIndex.py", line 85, in _index_object fdiff = difference(oldKeywords, newKeywords) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 25: ordinal not in range(128) =================================================================== This is a bit trickier to fix, as the variable fdiff that is calculated here is needed later on. But at this point I would like to ask: is this a solution direction I want to explore? Is the basic fix above sane? Or should we change nothing here and should add-on developers just be careful of what they let end up in the catalog? The fix/workaround from a user's point of view is: clear and rebuild the catalog as that gets rid of any old data so no comparison needs to be done anymore. That solves the problem for me. For reference, the PoiIssue from above was created when its class had this method (simplified): def Description(self): # return the contents of the details field return self.getRawDetails() And currently the code is this: def Description(self): details = self.getRawDetails() if not isinstance(details, unicode): encoding = getSiteEncoding(self) details = unicode(details, encoding) return details And that change means to solve another occasional unicode error when adding issues in Japanese: http://plone.org/products/poi/issues/135 I am the maintainer of Poi btw, and I am writing some migration code now that triggers this error. So writing some other migration to first fix that recatalog issue specifically for the Poi content is doable too. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ "This is your day, don't let them take it away." [Barlow Girl]

3 2

buildout-based buildbot back
by Christian Theune 06 Mar '08

06 Mar '08

Hi, it took a while, but here it is: I've put the buildout-based buildot into its own virtual machine (with enough resources hopefully). It's been running for about a week now and seems to work fairly well. I integrated the classical buildbot view and my alternative display in one installation that is accessible at: http://zopebuildout.whq.gocept.com - builds are run when the trunk of a buildout-based project is changed and at 3am CEST every day. - Updates for the project list are run before the nightly build and pick up all projects' trunks that define a buildout.cfg - Currently, all projects that don't define a test runner will appear as broken. I'll fix that soon so that non-existing test runners will be ignored. Christian -- gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - ct(a)gocept.com - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development

6 12

Zope Tests: 5 OK
by Zope Tests Summarizer 06 Mar '08

06 Mar '08

Summary of messages to the zope-tests list. Period Wed Mar 5 12:00:00 2008 UTC to Thu Mar 6 12:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Wed Mar 5 20:59:46 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009210.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Wed Mar 5 21:01:16 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009211.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Wed Mar 5 21:02:47 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009212.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Wed Mar 5 21:04:17 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009213.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Wed Mar 5 21:05:47 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009214.html

1 0

Dynamic formlibs
by Marcelo de Moraes Serpa 06 Mar '08

06 Mar '08

Hello, I have posted this message to the gepaid mailing list but nobody answered yet, so I thought I would give a try on this mailing list, as this issue is also related to Zope 3. Basically I have a fork of PloneGetPaid and did I am doing some major modification in it to adapt it to my business domain and requirements. In my modified version of PGP, the ShippingMethod and PaymentProcessor are not configured Store-wide, instead, they are configured per PayableItem. When a user creates a new payableitem (which in this particular framework is actually a custom content type and not regular content types marked by interfaces) the user can choose what Shipping Method and Payment Processor to use for this particular item. They are named adapters and I store their names in the object. What I would like to do (and have been trying to do without success) is to have the checkout forms be generated according to "metadata" defined on both the IShippingMethod and IPaymentMethod adapters. When the user chooses to buy a PayableItem, it goes straight to the checkout process (the checkout view is called in the payable item's view instead of the portal root). In the checkout-review-pay form's __init__ method, both named adapters are instantiated (using component.getAdapter (context,IShipphingMethod,name=context.getShipping_method()). There's a particular ShippingMethod that doesn't require any fields (only has some logic that is ran when the order is paid) . For this, I would like to show a warning in the form (arbitrary HTML) instead of fields, something like: "This product will be installed by our team, we will contact you with details" The User contact info and address are queried from the user model. Also for the PaymentProcessor, different payment processor require different fields or interface elements. My idea is to transfer the code that define, create and register the Transient bags, the code that define the fieldsets and widgets all to these adapters (IShippingMethod and IPaymentProcessor). However, I'm not sure yet on what to use to implement it. I thought on using viewlets, since it would add another layer of abstraction and allow me to insert html in them. Please, I really need some help here, what do you experienced getpaid developers think I should do in order to achieve this? I think that this idea could make into a good featureset for future getpaid releases! Thanks, Marcelo.

1 0

Zope Tests: 5 OK
by Zope Tests Summarizer 05 Mar '08

05 Mar '08

Summary of messages to the zope-tests list. Period Tue Mar 4 12:00:00 2008 UTC to Wed Mar 5 12:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Tue Mar 4 21:01:56 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009205.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Tue Mar 4 21:03:26 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009206.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Tue Mar 4 21:04:57 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009207.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Tue Mar 4 21:06:27 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009208.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Tue Mar 4 21:07:57 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009209.html

1 0

Looking for Google Summer of Code mentors!
by Martijn Faassen 04 Mar '08

04 Mar '08

Hi there, We are quickly approaching the Google of Summer of code again. The Zope Foundation's application needs to be in soon. In order to do a good application, we need to have a good selection of mentors. So, if you're a Zope developer and you want to mentor a student this summer, please add yourself to the list here: http://wiki.zope.org/gsoc/SummerOfCode2008 Don't wait: the Zope Foundation's application needs to be in *very* soon (by the 11th of march). The more mentors we have the better a chance we stand in getting projects accepted. Being a mentor isn't difficult and it can be very rewarding. All four of the students involved in the summer of code (Paul Carduner who worked on ajax for z3c.form, Nikhil N who worked on Zope 3 python 2.5 support, Uli Fouquet who worked on Grok's introspection UI and Luciano Ramalho who worked on a Grok-based application) are still active in our community, which we consider a great success! Being mentor is a privilege, and also a responsibility. We do expect mentors to stay in regular touch with their students. This means email and normal involvement on project mailing lists, but also in addition to this a weekly hour or so on irc and/or voice. We also have a few simple reporting requirements so that your mentor coordinator (yours truly) knows what's up and can solve any potential problems. Sign up and become a mentor! Everybody else (including mentors, students and others): * suggest projects, also on the same wiki page * suggest other mentors. Tell people who you *want* to be mentors to sign up. Everybody likes to be told they can be mentors, so tell them in public and get them to sign up. Students: * Mark the sign-up period on your calendar: it starts march 24. If we get accepted, then is the time to sign up. Regards, Martijn

2 2

Zope Tests: 5 OK
by Zope Tests Summarizer 04 Mar '08

04 Mar '08

Summary of messages to the zope-tests list. Period Mon Mar 3 12:00:00 2008 UTC to Tue Mar 4 12:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Mon Mar 3 21:01:15 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009200.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Mon Mar 3 21:02:46 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009201.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Mon Mar 3 21:04:16 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009202.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Mon Mar 3 21:05:48 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009203.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Mon Mar 3 21:07:18 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-March/009204.html

1 0

Developers wanted
by Zsembery Ágoston 04 Mar '08

04 Mar '08

Hello, We are looking forward to have a project with Zope 3. We are looking for Zope 3 contractors/developers to help us in the project. The project will last 4-5 months. All the work should be done remotely, at your own location. We may hire some people for a longer time. Please attach a short CV to your application and a desired rate. Please contact me directly if you are interested: Agoston ZSEMBERY agoston.zsembery(a)gmail.com

1 0

adding svn repositories for Zope 3 on Ohloh
by Paul Carduner 04 Mar '08

04 Mar '08

Those of you who know about the open source code analysis website ohloh.net may have noticed that Zope 3 has "Decreasing year-over-year development activity." We can change this status by adding the individual repository listings for all zope components (zope.app.*, zope.*, z3c.*, zc.*, etc.). I've started to add the few packages I've worked on to the ohloh code listings. Anyone with an ohloh login can do this by going to http://www.ohloh.net/projects/4495/enlistments and clicking on "Add a new enlistment". I encourage everyone to do this. -- Paul Carduner http://www.carduner.net

2 1

Zope 3's XML-RPC server: decode XML-RPC datatypes?
by Christian Theune 03 Mar '08

03 Mar '08

Hi, I was wondering that XML-RPC views receive date objects as XML-RPC types instead of datetime objects. For me this feels like a bug, but I'm unsure. Is this intentional or was it just overlooked? Christian -- gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - ct(a)gocept.com - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development

1 0