August 2004 - Zope-Dev

Running Zope tests in 2.7 fails
by Lennart Regebro 20 Aug '04

20 Aug '04

I have finally realized that the way tests are run in 2.7 is different from earlier version of Zope, and I'm trying to get them to run. It's Zope 2.7.2 on Debian. Python 2.3.4 and 2.3.2, so I guess 2.3.3 would give the same error. Could it be a side effect of the compiling or something? Here is the traceback: Running unit tests from /home/regebro/Zopes/Zope-2.7.2-0 /home/regebro/Zopes/Zope-2.7.2-0/lib/python/DocumentTemplate/cDocumentTemplate.so: undefined symbol: PyUnicodeUCS4_DecodeLatin1 ['/home/regebro/Zopes/Zope-2.7.2-0/lib/python', '/home/regebro/Zopes/Zope-2.7.2-0', '/usr/local/lib/python23.zip', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2', '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload', '/usr/local/lib/python2.3/site-packages'] Traceback (most recent call last): File "test.py", line 841, in ? process_args() File "test.py", line 831, in process_args bad = main(module_filter, test_filter, libdir) File "test.py", line 629, in main files = find_tests(module_filter) File "test.py", line 460, in find_tests os.path.walk(walkdir, finder.visit, rx) File "/usr/local/lib/python2.3/posixpath.py", line 290, in walk walk(name, func, arg) File "/usr/local/lib/python2.3/posixpath.py", line 290, in walk walk(name, func, arg) File "/usr/local/lib/python2.3/posixpath.py", line 282, in walk func(arg, top, names) File "test.py", line 439, in visit __import__(pkg) File "/home/regebro/Zopes/Zope-2.7.2-0/lib/python/AccessControl/__init__.py", line 16, in ? import DTML File "/home/regebro/Zopes/Zope-2.7.2-0/lib/python/AccessControl/DTML.py", line 18, in ? from DocumentTemplate import DT_Util File "/home/regebro/Zopes/Zope-2.7.2-0/lib/python/DocumentTemplate/__init__.py", line 22, in ? from DocumentTemplate import String, File, HTML, HTMLDefault, HTMLFile File "/home/regebro/Zopes/Zope-2.7.2-0/lib/python/DocumentTemplate/DocumentTemplate.py", line 112, in ? from DT_String import String, File File "/home/regebro/Zopes/Zope-2.7.2-0/lib/python/DocumentTemplate/DT_String.py", line 17, in ? from DT_Util import ParseError, InstanceDict, TemplateDict, render_blocks, str File "/home/regebro/Zopes/Zope-2.7.2-0/lib/python/DocumentTemplate/DT_Util.py", line 42, in ? from cDocumentTemplate import InstanceDict, TemplateDict, \ ImportError: /home/regebro/Zopes/Zope-2.7.2-0/lib/python/DocumentTemplate/cDocumentTemplate.so: undefined symbol: PyUnicodeUCS4_DecodeLatin1 Any ideas why this is? The Zope in itself seems to work fine. I have a 2.7.0 instance to, it also fails, but with a completely different error. //Lennart

3 3

Re: [Zope-dev] Patch: let non-seekable streams be input for ZPublisher (updated)
by Ames Andreas (MPA/DF) 20 Aug '04

20 Aug '04

Hi Dieter, *, Dieter Maurer wrote: > The "FieldStorage" must be rebuilt, because it can contain > file objects. These file objects must be reset as they may > have been (partially) read in the previous request. > This prevent reusing the previous "FieldStorage". As you may have seen in my first patch proposal I actually tried to address this problem, although halfbaken and buggy. So here's the next trial. I hope I've got it straight now. Why didn't I do something to reset the files, FiledStorage creates? Because I thought it wouldn't be needed. If you look at cgi.FieldStorage you will see, that accessing FieldStorage's 'value' field, will always reset the file before reading it and HTTPRequest's 'BODY' field does the same. Cause of your hint I looked again and realised that I was wrong. HTTPRquest's 'BODYFILE' field fails to reset the file before accessing it (as in OFS/Image.py the File class does). That could be easily fixed. What's worse is that FiledStorage can possibly create subinstances of itsellf which can possibly contain other fileobjects. This is the case for multipart-type requests. How to solve this? I can see three alternative aproaches: 1) In HTTPRequest.retry loop through self._fs.list (if it exists) and call seek(0) on every fileobject, you can find there. This can only work if the maximum depth of the FiledStorage tree is predictable, which in turn depends on the possible input. I don't know enough about the possible HTTP requests to determine such a limit or to even 'prove' its existance. 2) In HTTPRequest.retry recursively walk through the FieldStorage trees in self._fs.list (if it exists). I wouldn't do that in Python without being able to predict the (worst case) number of function calls needed to do this recursion. And without refreshing my knowledge of compiler techniques I wouldn't be able to roll the recursion out to a loop. 3) This is a somewhat drastic (and possibly conceived as hackish?) approach which OTOH is independent of the input and should perform quite well. I overwrite cgi.FieldStorage, esp. its make_file method, such that it adds all fileobjects, that it creates, to a (possibly caller provided) list. Thus the caller (i.e. HTTPRequest) has a list of all those fileobjects that need to be reset if the FieldStorage should be reused (as in ZPublisher's retries). For the patch attached I chose the third approach. Please let me know, what you think about it. If it is assessed to be ok I will update the collctor item, I opened yesterday. cheers, andreas

2 1

Re: [Zope-dev] Patch: let non-seekable streams be input for ZPublisher (updated)
by Ames Andreas (MPA/DF) 20 Aug '04

20 Aug '04

Hi, Andreas Ames wrote: > next trial. I hope I've got it straight now. Not quite so (although this may only be nitpicking ;-). fs_env = environ should have read fs_env = environ.copy() obviously. See attachment. cheers, andreas

1 0

Patch: let non-seekable streams be input for ZPublisher (updated)
by Ames Andreas (MPA/DF) 18 Aug '04

18 Aug '04

Hi, I had a thinko in my previous patch (http://mail.zope.org/pipermail/zope-dev/2004-August/023630.html) which is corrected in the attached version of the patch. Sorry for any inconvenience, I might have caused. As I'm sort of protocol challenged (thanks to corporate firewall), svn unfortunately isn't an option for me. Thus I created the patch against lib/python/ZPublisher/HTTPRequest.py as released in 2.7.2-0. As the patch is almost trivial it shouldn't be a problem to apply it even manually against other vesions (at least 2.7.0, 2.7.1 and 2.7.2 should work OOTB i.e. by using /usr/bin/patch). To sum up its purpose again: It's meant to avoid the stdin.seek() call in the HTTPRequest class so that non-seekable streams can be used as stdin. It also potentially avoids unnecessary (blocking) disk I/O in retries (this depends on the according request that's being worked on) and repeated request parsing in cgi.FieldStorage. The immediate cause for the patch is that AJPServer uses non-seekable streams as input for the ZPublisher. To change this locally I'd need to go with disk buffering. Nevertheless I think this could provide a general performance improvement. To this end I just save the cgi.FieldStorage instance, that is created by HTTPRequest.processInputs, when the request is worked upon for the first time, across retry boundaries. Questions: - Is this the right forum/place to send patches to? - Is there any chance that this could be applied to Zope's mainline? (If not I will proceed with a local disk buffering scheme in the long term.) cheers, andreas

3 2

Zope 2.7.2, restrictedTraverse or hasattr
by Kai Hoppert 18 Aug '04

18 Aug '04

Hi all, i actually have a Problem with restrictedTraverse and hasattr. Object1 Object2 Object3 I want to test if there exists a Object in Object3 wich has the same id and meta type how Object2 (id=myobject). so i ask Object3.restrictedTraverse('myobject',None) or hasattr(Object3,'myobject') i every time get's true back. I know that he jump's to Object2. i testet this with absolute_url, but this makes no sense i think. Either i never noticed that this both funktions works in this way or it is a bug. If it is no bug has anybody a solution for my problem? Greetings Kai Hoppert ___________________________________________________ tomcom Gesellschaft für Informationstechnologie mbH Heuriedweg 14 | D-88131 Lindau Fon +49 (0)8382 975844 | Fax +49 (0)8382 975855 kai.hoppert(a)tomcom.de | http://www.tomcom.de This document should only be read by those persons to whom it is addressed and is not intended to be relied upon by any person without subsequent written confirmation of its contents. tomcom disclaims all responsibility and accepts no liability for the consequences of any person acting, or refraining from acting on the contents of this document. Any unauthorised form of dissemination, copying, disclosure, modification, distribution and/or publication of this message is strictly prohibited. For information about tomcom please contact us on +49.8382.975844 or visit our web site at www.tomcom.de

3 2

Re: [Zope-dev] Re: "hasattr" geddon
by Shane Hathaway 18 Aug '04

18 Aug '04

On Saturday 10 July 2004 20:54 pm, Casey Duncan wrote: > Shane Hathaway <shane(a)hathawaymix.org> wrote: > > Why are > > we fixing hasattr and bare excepts when the real problem is ZODB? > > Well, the real problem is *probably* people, but I digress... http://c2.com/cgi/wiki?TheProcessIsTheProblem ;-) > Still, swallowing read conflicts can lead to bugs, even in read-only > tranactions. Consider the following function: > > def getStuffFrom(ob, default): > if hasattr(ob, 'stuff'): > return ob.stuff > return default > > I have seen this idiom about in some zope application/framework code. If > the hasattr results in a read conflict, then it returns false and > default is returned by the function even if ob does have the attribute. > In the example I saw (in DCWorkflow) this would result in the wrong UI > elements being displayed for ob mysteriously. Reloading the page would > likely fix the problem, baffling both the user and app developer, who > now both think Zope is playing with their minds. The change to ZODB that I'm suggesting would fix this. The suppressed conflict error would still mark the transaction as uncommitable. When the publisher calls commit() (which it does on every request even if nothing changed), it would result in a ConflictError, then the publisher would call abort() and the transaction would be retried. Users would never see results from a transaction that had any conflict. Shane

3 2

tal:on-error is a bare except...
by Florent Guillaume 17 Aug '04

17 Aug '04

tal:on-error is a bare except: !! Is there any opposition to adding an except ConflictError: raise in TALInterpreter::do_onError_tal ? Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 71 59 http://nuxeo.com mailto:fg@nuxeo.com

2 2

Disabled ViewCVS for the subversion repository.
by Jim Fulton 14 Aug '04

14 Aug '04

We've been having frequent cases of the berkeley databases used by our subversion repository becoming damaged and needeing recovery. Fortunately, it's easy to receover the database, but until it is recovered, subversion is unavailable. A while ago, I disabled tar downloads from ViewCVS and that made things much better. The database only got messed up every few weeks. :/ Lately, the problem has been getting worse. Some changes were made to ViewCVS to make better use of resources after I reported the problem with tar downloads. I decided to upgade to the latest version of ViewCVS on Thursday. (Unfortunately, the ViewCVS project no longer makes releases. The only way to get a version with subversion support is to use their CVS repository.) Since then, the problem seems to be much worse. I'm gonna try disabling ViewCVS for a while to see if the problem stops occuring without it. I want to verify that ViewCVS, and not something else, is the cause of the problem. Sorry for the inconvenience. Jim P.S. Yes, I'm aware that Subversion 1.1 has a file-system back end, and yes, I'm considering moving to it. I'm reluctant because 1) I have better things to do, 2) I'd rather wait for 1.1.1. -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

1 0

About populate folders by external scripts.
by Cristian S. Rocha 13 Aug '04

13 Aug '04

Hi, I'm working in a Product (CMFBio) to store biological data as Plone content. To begin the database I need to populate a Plone Folder with a lot of these contents (> 10000) in a batch way. That's the reason to make a little script who load a big file and create the objects in a folder. I was looking information about it, but was difficult to me found something in the zope.org page. I make a form to upload a file, but I would like do it in the command line. Could you help me with some examples to do that? Thanks, Cristian. -- Lic. Cristian S. Rocha <crocha(a)dc.uba.ar> Departamento de Computación, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires.

5 5

Adding a TemporaryFolder inside a factory method of a Zope class
by Paolo Bizzarri 13 Aug '04

13 Aug '04

Hi all. The scenario We would like to setup a Zope Class, subclassing the OFS.Folder, which has inside a cache. The structure would be something like: MyFolder TemporaryFolder MyFolderCache Therefore, the MyFolderCache cache is inside a temporary folder inside the MyFolder. I've no problem in manually creating this structure. However, we do have problems when we try to do this in a factory method. The factory method is something like: from Products.TemporaryFolder.TemporaryFolder import constructTemporaryFolder def manage_addMyFolderCache(container, id, title='') mfc = MyFolderCache(id, title) container._setObject(id, mfc) def manage_addMyFolder(container, id, title='') mf = MyFolder(id, title) container._setObject(id, mf) constructTemporaryFolder(mf, id='tmp', title='') manage_addMyFolderCache(mf.tmp, id='cache', title='cache') The Problem The problems occur inside manage_addMyFolderCache, when it tries the container._setObject. The error is always an AttributeError, because there is no _setObject. After a brief look at the MountedTemporaryFolder class, I can understand that there is still no folder inside mf.tmp, because the populate has not been called yet. Now the questions: 1)what is the problem which is occuring? 2) how can I do what I want to do in a single transaction? Thanks all for the support Regards Paolo -- Paolo Bizzarri - President - Icube S.r.l. Address: Via Ridolfi 15 - 56124 Pisa (PI), Italy E-mail: p.bizzarri(a)icube.it Web: http://www.icube.it Phone: (+39) 050 97 02 07 Fax: (+39) 050 31 36 588

3 3