- Zope-Dev - Zope lists

coverage.py, profile and hotshot support in Zope's testrunner
by Stefane Fermigier 24 Aug '05

24 Aug '05

I have added support for coverage analysis using coverage.py from Gareth Rees and Ned Batchelder, as well as support for profiling using either the profile or hotshot modules from the Python standard library, to Zope 2's test runner test.py. Justification: - coverage.py will let you interactively focus on the modules you want to check for coverage. - profile is slow to collect data but quick for analysis. - hotshot is fast when collecting but very slow for analysis. On the other hand, one can feed the great KCacheGrind tool with hotshot data after some transformation and this is the best way I have found so far for interactively exploring profile data. Since I'm not a regular Zope commiter, I'd like to ask for comments and permission first before commiting. The test.py file is here: http://blogs.nuxeo.com/sections/blogs/fermigier/2005_08_21_coverage-py-prof… BTW: the patch is absolutely trivial (see attached file). S. -- Stéfane Fermigier, Tel: +33 (0)6 63 04 12 77 (mobile). Nuxeo Collaborative Portal Server: http://www.nuxeo.com/cps Gestion de contenu web / portail collaboratif / groupware / open source! 1c1 < #!/home/fermigier/bin/python --- > #!/usr/bin/env python2.3 124,143d123 < --coverage < Use the coverage.py module from Gareth Rees < (http://www.nedbatchelder.com/code/modules/coverage.html) to collect data < for code coverage. This will output trace data in a file called < '.coverage'. You will need to call coverage.py after the test run to < analyse the data (-r for a line count report, -a for annotated file). < < --profile < Use the profile module from the standard library to collect profiling data. < This will output data in a file called '.profile'. You will need to use the < pstats module from the standard library after the test run to analyse the < data. < < --hotshot < Use the hotshot module from the standard library to collect profiling data. < This will output data in a file called '.hotshot'. You will need to use the < hotshot.pstats module from the standard library after the test run to < analyse the data. You may also use the hotshot2cg script from the < KCacheGrind sources to create data suitable for analysis by KCacheGrind. < 772,774d751 < COVERAGE = False < PROFILE = False < HOTSHOT = False 797,798c774 < "config-file=", "import-testing", < "coverage", "profile", "hotshot"]) --- > "config-file=", "import-testing"]) 850,855d825 < elif k == "--coverage": < COVERAGE = True < elif k == "--profile": < PROFILE = True < elif k == "--hotshot": < HOTSHOT = True 936,958d905 < < elif COVERAGE: < try: < from coverage import the_coverage < except: < print "You need to install coverage.py from " < print "http://www.nedbatchelder.com/code/modules/coverage.html" < sys.exit() < the_coverage.start() < main(module_filter, test_filter, libdir) < < elif PROFILE: < import profile < profile.runctx("main(module_filter, test_filter, libdir)", < globals=globals(), locals=vars(), < filename=".profile") < < elif HOTSHOT: < import hotshot < profile = hotshot.Profile(".hotshot") < profile.runctx("main(module_filter, test_filter, libdir)", < globals=globals(), locals=vars()) <

5 8

ZServer response RFC compliance improvement
by Jens Vagelpohl 23 Aug '05

23 Aug '05

Hi everyone, I'd like to get the zserver-content-length.patch createed by dunny in this collector issue: http://www.zope.org/Collectors/Zope/1866/collector_issue_contents merged into the Zope 2.7/2.8 branches and the trunk. It ensures that certain classes of responses (e.g. 304) correctly leave out the content-length header. Can anyone seee a valid reason for not merging it in? jens

3 3

Subtransaction backward compatibility
by Dieter Maurer 23 Aug '05

23 Aug '05

Subtransactions used to be used for two purposes: * ensure that newly created objects get "_p_" attributes (especially "_p_jar" and "_p_oid") * release memory in the mid of large transactions (i.e. reading and/or writing large amounts of objects) With ZODB 3.4, subtransactions are implemented as savepoints. They can still be used for the first purpose. But, they no longer start cache garbage collection. As a consequence, subtransactions/savepoints can be dropped at places where only the second purpose has mattered, e.g. in Zope's ZCatalog. -- Dieter

2 1

RE: [Zope-dev] Python 2.4 with zope 2.7.6, on Windows
by Aruna Kathiria 23 Aug '05

23 Aug '05

Yeah, it's true that I have to use Python 2.4 with Zope 2.7.6 Also I can't go for Zope 2.8.1. I also tried to fix few places but still cannot. Dieter, do you remember what and where you had to fix? Thanks.. With Regards, Aruna Kathiriya Sr.Consultant, CIGNEX Technologies, Inc T: 408.327.9900 x 306 F: 408.273.6785 E: aruna(a)cignex.com U: www.cignex.com "Implement IT Right" -----Original Message----- From: Dieter Maurer [mailto:dieter@handshake.de] Sent: Monday, August 22, 2005 9:56 AM To: Aruna Kathiria Cc: zope-dev(a)zope.org Subject: Re: [Zope-dev] Python 2.4 with zope 2.7.6, on Windows Aruna Kathiria wrote at 2005-8-21 23:53 -0700: >Is it possible to use Python 2.4 with Zope 2.7.6 on windows plateform? We use Python 2.4 with Zope 2.7.2. However (!) I had to fix a few places (identified by failing test cases from Zope quite extensive test suite). Apparently, Zope 2.8.1 is better prepared for Python 2.4. And others already told you: you use Python 2.4 with Zope at your own risk! -- Dieter

2 1

Python 2.4 with zope 2.7.6, on Windows
by Aruna Kathiria 22 Aug '05

22 Aug '05

Hello Everyone: Is it possible to use Python 2.4 with Zope 2.7.6 on windows plateform? I come to know how to use them on UNIX from /doc/INSTALL.txt but don't know how to install and use Python 2.4 and Zope 2.7.6 on windows platform. By default Zope 2.7.6 is bundled with Python 2.3.5. For some reasons I want to use Python 2.4 with Zope 2.7.6 on Windows. Thanks in advance. With Regards, Aruna Kathiriya Sr.Consultant, CIGNEX Technologies, Inc T: 408.327.9900 x 306 F: 408.273.6785 E: aruna(a)cignex.com U: www.cignex.com "Implement IT Right"

3 2

OSX: zopectl test results in "Interrupted system call"
by Stefan H. Holek 21 Aug '05

21 Aug '05

Since r37701 "zopectl test" forks off a child process [1]. On OSX I now reliably get a traceback tacked onto the end of every test report: Traceback (most recent call last): File "/Zope8/lib/python/Zope2/Startup/zopectl.py", line 312, in ? main() File "/Zope8/lib/python/Zope2/Startup/zopectl.py", line 281, in main c.onecmd(" ".join(options.args)) File "/usr/local/python2.3/lib/python2.3/cmd.py", line 210, in onecmd return func(arg) File "/Zope8/lib/python/Zope2/Startup/zopectl.py", line 266, in do_test os.waitpid(pid, 0) OSError: [Errno 4] Interrupted system call Tested on two OSX machines. Appears to work fine on Linux. Anybody else seeing this? Ideas? Thanks, Stefan [1] <http://svn.zope.org/Zope/branches/Zope-2_8-branch/lib/python/ Zope2/Startup/zopectl.py?rev=37701&r1=30475&r2=37701> -- Anything that happens, happens. --Douglas Adams

2 1

Windows builds
by Christian Theune 17 Aug '05

17 Aug '05

Hi, I announced myself stepping back from doing the windows builds some while ago. Although nobody stepped up in taking this task, I'll make a hard cut right now. I'm preparing the 2.8.1 builds right now and those will definitely be the last windows builds prepared by me. Two reasons for the stepping back: time constraints from my side and technical unability to safely create the windows builds. Cheers, Christian -- Tagung der Deutschsprachigen Zope Usergroup in Köthen 22. und 23. September 2005 -- Details auf http://www.dzug.org ct(a)gocept.com -- gocept gmbh & co. kg -- +49 3496 30 99 112

3 2

BTrees.Length conflict resolution
by Martijn Pieters 12 Aug '05

12 Aug '05

Did the conflict resolution code for BTrees.Length ever work? Because as it stands now the code will fail as it assumes that integers are passed in, instead of state dictionaries: def _p_resolveConflict(self, old, s1, s2): return s1 + s2 - old As there are no tests for this that I can see (the BTrees tests are kinda very dense), I am not too keen to go touch this, but I think this should read: def _p_resolveConflict(self, old, s1, s2): s1['value'] += s2['value'] - old['value'] return s1 Martijn Pieters, up to his armpits in conflict resolution code.

2 2

Catalog and Unicode
by Florent Guillaume 11 Aug '05

11 Aug '05

We're seeing problems in one application here due to the catalog and interactions with Unicode. Here's what happens: - an object is indexed with a Unicode title, so in the catalog the metadata tuple has for instance (u'cafe',) - later that title is changed to latin-1, so the new metadata tuple would be ('caf\xe9',) The problem is that Catalog.py has in updateMetadata() the code: if data.get(index, 0) != newDataRecord: data[index] = newDataRecord The simple comparison in the first line provokes a UnicodeDecodeError, you can reproduce by a simple: python -c "u'e' == '\xe9'" This understandable, but in the case of the catalog really not helpful. I propose to change the code above to: try: changed = data.get(index, 0) != newDataRecord except UnicodeDecodeError: changed = True if changed: data[index] = newDataRecord Objections ? Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

2 2

Is there a Zope 2.7.8 planned?
by Paul Winkler 10 Aug '05

10 Aug '05

Hiya, I'm looking at the two patches that were uploaded for http://www.zope.org/Collectors/Zope/1810 which is assigned to me... BUT: 1) Have we reached the end of the 2.7 line? Do I need to apply this to the 2.7 branch? Or can I just put this on the trunk and 2.8 branches and leave it at that? I really hate having to deal with both CVS and SVN for one fix... 2) Since 2.8.1 is planned for tomorrow, should I a) hurry up and get it in before that, or b) wait because I don't want to risk merging a not-widely-tested patch so close to the final release? or c) just create a bugfix branch and leave it for somebody else to sort out the merge, since I'll be out of town for the next 2 weeks starting tomorrow and can't help if there's anything wrong with the fix? :-) thanks, PW

3 3