November 2005 - Zope-Dev

Re: SVN: Zope/trunk/lib/python/Products/PythonScripts/tests/testPythonScript.py Suppress expected DeprecationWarning output in test.
by Florent Guillaume 23 Nov '05

23 Nov '05

Isn't a try: ... finally: ... needed here? Florent Tres Seaver wrote: > Log message for revision 40329: > Suppress expected DeprecationWarning output in test. > > Changed: > U Zope/trunk/lib/python/Products/PythonScripts/tests/testPythonScript.py > > -=- > Modified: Zope/trunk/lib/python/Products/PythonScripts/tests/testPythonScript.py > =================================================================== > --- Zope/trunk/lib/python/Products/PythonScripts/tests/testPythonScript.py 2005-11-22 18:18:17 UTC (rev 40328) > +++ Zope/trunk/lib/python/Products/PythonScripts/tests/testPythonScript.py 2005-11-22 18:18:36 UTC (rev 40329) > @@ -25,6 +25,30 @@ > if not here: > here = os.getcwd() > > +class WarningInterceptor: > + > + _old_stderr = None > + _our_stderr_stream = None > + > + def _trap_warning_output( self ): > + > + if self._old_stderr is not None: > + return > + > + import sys > + from StringIO import StringIO > + > + self._old_stderr = sys.stderr > + self._our_stderr_stream = sys.stderr = StringIO() > + > + def _free_warning_output( self ): > + > + if self._old_stderr is None: > + return > + > + import sys > + sys.stderr = self._old_stderr > + > # Test Classes > > def readf(name): > @@ -219,7 +243,7 @@ > f = self._newPS(defn + "\n" + asn % name) > self.assertRaises(TypeError, f) > > -class TestPythonScriptGlobals(PythonScriptTestBase): > +class TestPythonScriptGlobals(PythonScriptTestBase, WarningInterceptor): > def _exec(self, script, bound_names=None, args=None, kws=None): > if args is None: > args = () > @@ -241,8 +265,10 @@ > def test_filepath(self): > # This test is meant to raise a deprecation warning. > # It used to fail mysteriously instead. > + self._trap_warning_output() > f = self._filePS('filepath') > self.assertEqual(f(), [0]) > + self._free_warning_output() > > def test_suite(): > suite = unittest.TestSuite() > > _______________________________________________ > Zope-Checkins maillist - Zope-Checkins(a)zope.org > http://mail.zope.org/mailman/listinfo/zope-checkins > -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

2 1

RE: Memory Leak Question
by Dan Pozmanter 22 Nov '05

22 Nov '05

Sure thing: I was mucking around in _Acquisition.c with getattr, findattr, etc, and I noticed checked to see if my code was leaky. It was! I then checked just vanilla code, and found the same leak (only much much smaller). So here is what I did to make things more visible: In Wrapper_getattro, I simply added a while loop, and another call to Wrapper_findattr before the return statement " return Wrapper_findattr(self, oname, NULL, NULL, NULL, 1, 1, 0, 0);". Loop it 1000 times for some fireworks, 5 times for a more modest display. Looping other calls (such as Wrapper_acquire before " return Wrapper_acquire(self, oname, filter, extra, orig, explicit, containment);" in Wrapper_findattr) does not produce a similar leak. So. To reproduce: Edit _Acquisition.c Function Wrapper_getattro add " int i = 0;" at the top. then, if self->obj ... and all that, add: " while (i < xyz) { Wrapper_findattr(self, oname, NULL, NULL, NULL, 1, 1, 0, 0); i++; } " where xyz is an integer of your choice. Simply go to the index_html the root or any other object, fire up task manager (I produced this bug on windows), then refresh that page rigorously. Note that this bug can be reproduced without modifying any code. The loop simply amplifies the leak, making it easier to spot. (Or the loop causes huge additional problems I am not aware of. In any case, doing *something* clever to call Wrapper_findattr more than once should make the problem easier to spot.) _______________ Daniel Pozmanter Siteworx, Inc. "Festina Lente" - Gaius Julius -----Original Message----- From: Florent Guillaume [mailto:fg@nuxeo.com] Sent: Monday, November 07, 2005 3:30 PM To: Dan Pozmanter Cc: zope-dev(a)zope.org Subject: Re: Memory Leak Question Could you provide details about why you think there's a leak in the code (what led you to Wrapper_findattr), and how to reproduce it consistently on a blank platform (Zope 2.8.4 preferred)? Maybe open a bug in the collector at http://www.zope.org/Collectors/Zope/ also. Thanks, Florent Dan Pozmanter wrote: > Consider it reported then. > (Sorry for the delay, I was out of the office for most of last week). > > It looks like the problem is in Wrapper_findattr. > > Any thoughts? > > _______________ > > Daniel Pozmanter > Siteworx, Inc. > > "Festina Lente" - Gaius Julius > > -----Original Message----- > From: Chris McDonough [mailto:chrism@plope.com] > Sent: Monday, October 31, 2005 10:33 PM > To: Dan Pozmanter > Cc: zope-dev(a)zope.org > Subject: Re: [Zope-dev] Memory Leak Question > > Nobody has yet reported a memory leak symptom against any particular > piece of Zope 2.8.X yet, AFAIK. > > On Oct 31, 2005, at 6:34 PM, Dan Pozmanter wrote: > > >>Ahoy, >> >> I was poking around in _Acquisition.c, and noticed my memory usage >>spiked. >>Returning to the vanilla code (2.8.2-final), I noticed the same spike >>(just less pronounced). >>(If you refresh the page a few gazillion times this pops up). >> >>I would not categorize this as an actual bug report (I'd like to test >>more rigorously with a script to verify a leak), but after reading the > > >>note on Changes for 2.8.x in the to do section, I was curious: >>Is this a known issue? >> >>If so, has there been any headway, and can I be of any assistance in >>resolving it? >> -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

2 1

Re: Memory Leak, and Changing Severity of a Bug
by Tres Seaver 22 Nov '05

22 Nov '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dan Pozmanter wrote: > Yes, I saw the follow up. > > So is there no way to change the severity of a bug? It could be changed, but you need to persuade us that it is a "showstopper" first. Labeling the bug "critical" is a claim that we have to fix it before making another release, which isn't reasonable without a reproducible test case. > Back to the bug. > > ''' > I was poking around in _Acquisition.c, and noticed my memory usage > spiked. > Returning to the vanilla code (2.8.2-final), I noticed the same spike > (just less pronounced). > (If you refresh the page a few gazillion times this pops up). > ''' > In other words, this is happening in utterly unmodified c code. > > Thanks for the link, I'll do some poking around. > Is there a standard way for freeing the memory for the result of a > Wrapper_findattr call from within _Acquistion.c? See http://docs.python.org/api/countingRefs.html Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver(a)palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDg5O++gerLs4ltQ4RArzfAKCzwxMi1sA3iHfT3ksQOPqYPv8wZgCfZ08T 2L9r5Ju8cnw51VZIEpgNYXQ= =z2el -----END PGP SIGNATURE-----

1 0

Memory Leak, and Changing Severity of a Bug
by Dan Pozmanter 22 Nov '05

22 Nov '05

I'd like to upgrade bug 1950 to critical. Is there an easy way to do this? _______________ Daniel Pozmanter Siteworx, Inc. "Festina Lente" - Gaius Julius

2 1

zope.testrecorder
by Benji York 22 Nov '05

22 Nov '05

We don't have the collective time right now to "market" this, but it's very cool so we thought someone might get some use out of it. Brian Lloyd has developed zope.testrecorder, a cross-browser (IE, Firefox, Safari) JavaScript app that records browser events (clicking, entering text, etc.) and test assertions (this text box should contain this text, this checkbox should be unchecked, etc.) and renders the captured events as either an HTML Selenium test or a doctest using zope.testbrowser. It can be installed as a product into Zope 2 or as a Python package in Zope 3. The rendering framework is pluggable and it's pretty easy to add new rendering options for other testing systems (twill, etc.). The testrecorder is currently in "beta" form, but is very handy, so if anyone wants to use/market/improve/add tests for it, go ahead. :) Get it via Subversion at svn://svn.zope.org/repos/main/zope.testrecorder/trunk -- Benji York Senior Software Engineer Zope Corporation

1 0

Running all unittests in $INSTANCE_HOME
by Sidnei da Silva 22 Nov '05

22 Nov '05

I might have asked this before, but can't seem to find it. I'm looking for a way to run all tests in $INSTANCE_HOME using the test.py script, but can't seem to get the majik combination of parameters together. Would anyone hint me? -- Sidnei da Silva Enfold Systems, LLC. http://enfoldsystems.com

5 10

Re: Hard-coded "Content-type:text/xml" as xmlrpc in Zope (2.8.1)
by Florent Guillaume 22 Nov '05

22 Nov '05

[redirected to zope-dev] John Ziniti wrote: > Back in June 2004, I had the following problem: > > http://mail.zope.org/pipermail/zope/2004-June/151436.html > http://mail.zope.org/pipermail/zope/2004-June/151492.html > http://mail.zope.org/pipermail/zope/2004-June/151497.html > http://mail.zope.org/pipermail/zope/2004-June/151498.html > > which came about because HTTPRequest.py was hard-coded > to assume that any request with a Content-type of "text/xml" > is an xmplrpc call and handles the request as such. > > I resolved this issue with the help of brian-CWUwpEBWKX0(a)public.gmane.org, who > gave me the XmlFix product, which monkey-patches the > HTTPRequest.processInputs method to respond a bit more > appropriately. We noted in the thread that although Zope's > behavior is broken, it is unlikely to change for > historical reasons. I am fine with this and felt comfortable > with the simplicity and supportability of the monkey-patch. > > This monkey-patch was no longer effective recently when I > upgraded our Zope instances to 2.8.1. > > Zope-2.8.1 now additionally has the zope.app.publication. > HTTPPublicationRequestFactory class, which also assumes > that "text/xml" means xmlrpc (in fact, it assumes that > anything that startswith('text/xml') is an xmlrpc call). > > Not fixing old code which acts in strange hard-coded ways is > something I understand, but adding new code that also > acts that way seems like it is getting further away > from "good" :) > > Can anything be done to change the behavior in the > HTTPPublicationRequestFactory so that it is not so > obtuse? My monkey-patch now has to munge two methods > to make xml uploads work and I am less happy with it > than I was before. This could be turned into a "disable-xmlrpc" config option. Could you provide your updated monkey-patch? Florent -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

1 0

Bad security declarations
by Florent Guillaume 21 Nov '05

21 Nov '05

I've added a long-needed warning if you attempt to make a security declaration for a nonexistent method, usually because of typos. Checked in for 2.8, 2.9 and trunk. Look carefuly for WARNING in your logs, there are already a few bad ones in CMF (I don't have time to fix them tonight though): > WARNING Init Class Products.CMFCore.PortalContent.PortalContent has > a security declaration for nonexistent method 'manage_FTPget' > WARNING Init Class Products.CMFSetup.tool.SetupTool has a security > declaration for nonexistent method 'getImportContextId' > WARNING Init Class Products.CMFSetup.tool.SetupTool has a security > declaration for nonexistent method 'runAllSetupSteps' > WARNING Init Class Products.CMFSetup.tool.SetupTool has a security > declaration for nonexistent method 'executeStep' Florent -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

2 1

Re: Zope 2 security and SimpleItem
by Florent Guillaume 20 Nov '05

20 Nov '05

[Intended for zope-dev actually...] Florent Guillaume wrote: > Florent Guillaume wrote: > >> I'm in the process of refactoring OFS to use new-style security >> declarations (about time ;)), and I stumbled on something which may >> or may not be a bug, I don't know, I'd like some else's opinion: >> >> The class SimpleItem has the definition (it's been there since the >> begining of time when SimpleItem was created): >> __ac_permissions__=(('View', ()),) >> The unusual thing here is () instead of ('',). > > > Hm I now realize that this may be there just to define the View > permission as available but that's all. But there's still a discrepancy > in the way SecurityInfo treats it. Ok I got to the reasons for that, it all dates back to the origins of SecurityInfo in December 2000 when it originally had methods that did the work of both declareProtected and declareObjectProtected. I'll fix this. Florent > > Anyway I'm further in understanding securiy in Z2 than I've ever been :) > > Florent > > >> I think the intent here is that it be the object level protection, >> equivalent to the modern declareObjectProtected('View'). >> Indeed, if the SimpleItem class had a >> security = ClassSecurityInfo() >> (even by itself without further security declaration), then >> AccessControl/SecurityInfo, which has the code >> # Empty names list sets access to the class itself, named '' >> if not len(names): >> names = ('',) >> would actually turn the () into a ('',) and the rest of the end of >> the security setup, in App/class_init.py, would set >> SimpleItem.__roles__ = PermissionRole('View') and that would be it. >> >> However SimpleItem does *not* have this ClassSecurityInfo, which >> means that the code above is not called, and the final logic in >> class_init.py does not turn an empty tuple into "protect the object". >> >> It means that SimpleItem does not have an object level protection of >> View (but the default which is that only Manager has access), which >> is probably as well but not clear from the code. >> >> What do you think I should do? >> - fix to use View? >> - fix to use nothing? >> >> Florent >> > > -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg-CQNGlVZ5l00AvxtiuMwx3w(a)public.gmane.org _______________________________________________ Zope-CMF maillist - Zope-CMF-lEa0QfImRKZ5o+NzvwT5Tw(a)public.gmane.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

2 1

Zope 2 security and SimpleItem
by Florent Guillaume 20 Nov '05

20 Nov '05

I'm in the process of refactoring OFS to use new-style security declarations (about time ;)), and I stumbled on something which may or may not be a bug, I don't know, I'd like some else's opinion: The class SimpleItem has the definition (it's been there since the begining of time when SimpleItem was created): __ac_permissions__=(('View', ()),) The unusual thing here is () instead of ('',). I think the intent here is that it be the object level protection, equivalent to the modern declareObjectProtected('View'). Indeed, if the SimpleItem class had a security = ClassSecurityInfo() (even by itself without further security declaration), then AccessControl/SecurityInfo, which has the code # Empty names list sets access to the class itself, named '' if not len(names): names = ('',) would actually turn the () into a ('',) and the rest of the end of the security setup, in App/class_init.py, would set SimpleItem.__roles__ = PermissionRole('View') and that would be it. However SimpleItem does *not* have this ClassSecurityInfo, which means that the code above is not called, and the final logic in class_init.py does not turn an empty tuple into "protect the object". It means that SimpleItem does not have an object level protection of View (but the default which is that only Manager has access), which is probably as well but not clear from the code. What do you think I should do? - fix to use View? - fix to use nothing? Florent -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

1 1