Re: Clarification re: Zope X3.1, 2.8
Before I vanished for PyCon about two weeks ago, I was under the impression that merging Zope/branches/five-integration into Zope/trunk was imminent -- a matter of days, if not hours. Perhaps I was mistaken in that. Regardless, what's the current status of this? Last I saw, Andreas announced a plan to release 2.8a2 this coming Friday evening (according to my clock <wink>), but AFAICT the five-integration branch is still distinct from the trunk. Is it the plan to release 2.8a2 without merging this in? If not, what's the plan/schedule for merging five-integration? ZODB 3.4 requires some Zope3 packages, so there are potential integration headaches in two directions: (1) possible problems for ZODB due to five-integration using versions of those packages older than the ones ZODB 3.4 has been using; and, (2) code in five-integration that runs afoul of changes in ZODB 3.4. I can't predict these without trying the actual code. A possible example of #2 is that I rewrote hundreds of instances of get_transaction() in Zope trunks yesterday, because get_transaction() is officially deprecated in ZODB 3.4 and I don't want to release a Zope that raises DeprecationWarning in its internals. Perhaps the five-integration code has more instances of this that need to be changed. A clarification, because some have been confused about this (and it is confusing!): a ZODB trunk checkout stitches in its own copies of the Zope3 packages it needs, same as it stitches in copies of zdaemon and ZConfig. The process of stitching a ZODB into a Zope does *not* stitch those packages into that Zope: Zope uses its own copies of zdaemon, ZConfig, and Zope3 code. From Zope 2.8's POV, "ZODB" means exactly these 9 directories (these, and no others, get stitched in from a ZODB tag): Under lib/python/: ZODB ZEO persistent transaction BTrees ThreadedAsync Persistence ZopeUndo Under utilities/: ZODBTools (this is a renamed copy of ZODB's src/scripts/). I always expected (and still do) to do the ZODB stitching on Zope trunk, but since stitching in a ZODB does not include anything other than those 9 directories, I can't do it before at least the Zope3 packages are stitched in from the five-integration branch. Note that it's also possible we'll hit integration snags due to mismatches in the versions of zdaemon and ZConfig in use (the ones Zope trunk uses versus the ones ZODB has been using).
--On Dienstag, 29. März 2005 14:07 Uhr -0500 Tim Peters <tim@zope.com> wrote:
Regardless, what's the current status of this? Last I saw, Andreas announced a plan to release 2.8a2 this coming Friday evening (according to my clock <wink>), but AFAICT the five-integration branch is still distinct from the trunk.
Right. My plan was to post a reminder that 2.8a2 is scheduled for this weekend. So all Five related integration work should be finished very soon otherwise I am going to reschedule a2.
ZODB 3.4 requires some Zope3 packages, so there are potential integration headaches in two directions: (1) possible problems for ZODB due to five-integration using versions of those packages older than the ones ZODB 3.4 has been using; and, (2) code in five-integration that runs afoul of changes in ZODB 3.4. I can't predict these without trying the actual code.
I hope that the people doing the Five integration should have their stuff ready in time but there is no need to hurry with a release just for the sake of making a release. Means: if you need more time you weill get the time of course. Andreas
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andreas Jung wrote:
--On Dienstag, 29. März 2005 14:07 Uhr -0500 Tim Peters <tim@zope.com> wrote:
Regardless, what's the current status of this? Last I saw, Andreas announced a plan to release 2.8a2 this coming Friday evening (according to my clock <wink>), but AFAICT the five-integration branch is still distinct from the trunk.
Right. My plan was to post a reminder that 2.8a2 is scheduled for this weekend. So all Five related integration work should be finished very soon otherwise I am going to reschedule a2.
Most of that work has been done on the trunk. The 'five-integration' branch changes consist largely of: - Setting up an 'svn:external' link to the Zope X3 3.0 repository (which is on a branch, pruned to include only the packages which were actually released with 3.0). - Importing the Five product. - Un-monkey-fying Five's monkey patches. As soon as we settle the question of how ZODB gets stitched in (I prefer the 'svn:external' mode myself, but that is just a preference), that branch should be easily merged.
ZODB 3.4 requires some Zope3 packages, so there are potential integration headaches in two directions: (1) possible problems for ZODB due to five-integration using versions of those packages older than the ones ZODB 3.4 has been using; and, (2) code in five-integration that runs afoul of changes in ZODB 3.4. I can't predict these without trying the actual code.
I hope that the people doing the Five integration should have their stuff ready in time but there is no need to hurry with a release just for the sake of making a release. Means: if you need more time you weill get the time of course.
Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCScNFGqWXf00rNCgRAixAAJwIHhVuNkhLzQsYK9UAzXeKrLTKhgCfeY/9 hBobU0eAnoBgZHdf91yMcfw= =dLdd -----END PGP SIGNATURE-----
[Tres Seaver] ...
Most of that work has been done on the trunk. The 'five-integration' branch changes consist largely of:
- Setting up an 'svn:external' link to the Zope X3 3.0 repository (which is on a branch, pruned to include only the packages which were actually released with 3.0).
Just noting that this may create new but short-lived problems for developers on Windows (can't tell for sure until I try it; has to do with PuTTY's "symbolic names", and the unlikelihood that any Windowshead has "svn.zope.org" set up as a symbolic name now).
- Importing the Five product.
- Un-monkey-fying Five's monkey patches.
As soon as we settle the question of how ZODB gets stitched in (I prefer the 'svn:external' mode myself, but that is just a preference), that branch should be easily merged.
Since Zope doesn't "own" ZODB, I don't see how merging the branch has anything to do with how ZODB gets stitched in. Stitching in a _new_ ZODB is my problem, after the branch is merged (& I need the merge to happen first, so that Zope3 (lib/python/zope) packages show up on the trunk -- ZODB 3.4 can't work without them). AFAICT, no stitching of ZODB took place when five-integration branch was created. For example, the log message for Zope/branches/five-integration/lib/python/ZODB just says it was copied from Zope/trunk:29468. IOW, it just copied over the version of ZODB that happened to be in Zope trunk at the time the five-integration branch was created. If people then made _changes_ to ZODB code on five-integration, they should not have, and it will create problems. But if they didn't, the ZODB code on five-integration branch and Zope trunk should still be identical, in which case no code in any of the 9 ZODB directories should have any effect on the merge. IOW, ignore ZODB entirely: if the tests pass on the branch, they should continue to pass on the trunk after the merge, since the ZODBs on branch and trunk should be exactly the same right now.
... [Tres]
- Setting up an 'svn:external' link to the Zope X3 3.0 repository (which is on a branch, pruned to include only the packages which were actually released with 3.0).
[Tim]
Just noting that this may create new but short-lived problems for developers on Windows (can't tell for sure until I try it; has to do with PuTTY's "symbolic names", and the unlikelihood that any Windowshead has "svn.zope.org" set up as a symbolic name now).
Bit o' good news: there doesn't appear to be a Windows problem here (might have been if we were using svn+ssh:, but not with plain svn: access). ...
IOW, ignore ZODB entirely: if the tests pass on the branch, they should continue to pass on the trunk after the merge, since the ZODBs on branch and trunk should be exactly the same right now.
Using a fresh checkout: """ five\lib\python$ svn log -rHEAD:0 -v --stop-on-copy ZODB ZEO ^ persistent transaction BTrees ThreadedAsync ^ Persistence ZopeUndo ------------------------------------------------------------------------ r29469 | efge | 2005-03-15 06:23:03 -0500 (Tue, 15 Mar 2005) | 2 lines Changed paths: A /Zope/branches/five-integration (from /Zope/trunk:29468) Branch where integration of Five will take place. ------------------------------------------------------------------------ five\lib\python$ cd ..\..\utilities five\utilities$ svn log -rHEAD:0 -v --stop-on-copy ZODBTools ------------------------------------------------------------------------ r29469 | efge | 2005-03-15 06:23:03 -0500 (Tue, 15 Mar 2005) | 2 lines Changed paths: A /Zope/branches/five-integration (from /Zope/trunk:29468) Branch where integration of Five will take place. ------------------------------------------------------------------------ """ IOW, no changes have been checked in to any ZODB code on five-integration branch, so it should indeed be identical to the ZODB code still on Zope trunk. The same appears true of zdaemon and ZConfig too, and AFAIK that covers all the "external" code stitched in to Zope apart from new external code introduced on the five-integration branch.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tim Peters wrote:
[Tres Seaver] ...
Most of that work has been done on the trunk. The 'five-integration' branch changes consist largely of:
- Setting up an 'svn:external' link to the Zope X3 3.0 repository (which is on a branch, pruned to include only the packages which were actually released with 3.0).
Just noting that this may create new but short-lived problems for developers on Windows (can't tell for sure until I try it; has to do with PuTTY's "symbolic names", and the unlikelihood that any Windowshead has "svn.zope.org" set up as a symbolic name now).
I'll have to take your word for that; are you saying that 'svn:external' doesn't work by default in the windows SVN clients?
- Importing the Five product.
- Un-monkey-fying Five's monkey patches.
As soon as we settle the question of how ZODB gets stitched in (I prefer the 'svn:external' mode myself, but that is just a preference), that branch should be easily merged.
Since Zope doesn't "own" ZODB, I don't see how merging the branch has anything to do with how ZODB gets stitched in. Stitching in a _new_ ZODB is my problem, after the branch is merged (& I need the merge to happen first, so that Zope3 (lib/python/zope) packages show up on the trunk -- ZODB 3.4 can't work without them).
OK, that works for me. AFAIK, the branch should be ready to merge "whenever"; all the recent work on it has been to merge fixes which had already been applied on the trunk. How does this sound for a recipe: - ReleaseMaker merges the 'five-integration' branch to the trunk, and tests it. - ZODBGuru uses 'svn rm' to zap the current ZODB on the trunk, replacing it with an 'svn:external' link to your ZODB 3.4 tag; and then tests it (could be an 'svn cp', but I don't see any benefit to maintaining a Zope-specific fork). - ReleaseMaker updates changelog, version.txt, etc. and checks in. - ReleaseMaker uses 'svn cp' to create the release tag for Zope 2.8a2 (whatever it is being called).
AFAICT, no stitching of ZODB took place when five-integration branch was created. For example, the log message for Zope/branches/five-integration/lib/python/ZODB just says it was copied from Zope/trunk:29468. IOW, it just copied over the version of ZODB that happened to be in Zope trunk at the time the five-integration branch was created.
If people then made _changes_ to ZODB code on five-integration, they should not have, and it will create problems.
The goal for that branch was to do *only* stuff which had to do with landing Five / ZopeX3.0; no other changes were supposed to land there. Any non-Five-specific work was supposed to happen on the trunk.
But if they didn't, the ZODB code on five-integration branch and Zope trunk should still be identical, in which case no code in any of the 9 ZODB directories should have any effect on the merge.
IOW, ignore ZODB entirely: if the tests pass on the branch, they should continue to pass on the trunk after the merge, since the ZODBs on branch and trunk should be exactly the same right now.
They should be identical, as none of the checkins on the 'five-integration' branch touched anything under ZODB. Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCSd2gGqWXf00rNCgRAkPkAJ4gZhGlb5sDAr0QlDtOPJ3N4BE+pQCfdf4W IUamIwQMwMO6HiD9YWdiVoI= =SyzX -----END PGP SIGNATURE-----
[Tres] ...
I'll have to take your word for that; are you saying that 'svn:external' doesn't work by default in the windows SVN clients?
Crossed in the mail; no problem here. ...
OK, that works for me. AFAIK, the branch should be ready to merge "whenever"; all the recent work on it has been to merge fixes which had already been applied on the trunk. How does this sound for a recipe:
- ReleaseMaker merges the 'five-integration' branch to the trunk, and tests it.
Check.
- ZODBGuru uses 'svn rm' to zap the current ZODB on the trunk, replacing it with an 'svn:external' link to your ZODB 3.4 tag; and then tests it (could be an 'svn cp', but I don't see any benefit to maintaining a Zope-specific fork).
My problem. It will certainly be done via 9 "svn switch"s on my local machine first. As explained elsewhere, there are several other possible sources of integration problems here, and I can't know about those before actually trying it. It's possible, e.g., that I'll need to change ZODB 3.4 to worm around them, or switch ZODB to using a different ZConfig, etc. Can't predict here.
- ReleaseMaker updates changelog, version.txt, etc. and checks in.
Check.
- ReleaseMaker uses 'svn cp' to create the release tag for Zope 2.8a2 (whatever it is being called).
Check. ....
The goal for that branch was to do *only* stuff which had to do with landing Five / ZopeX3.0; no other changes were supposed to land there. Any non-Five-specific work was supposed to happen on the trunk.
Crossed in the mail again; I confirmed this is so for the zdaemon, ZConfig and nine ZODB directories. ... Just ran the five-integration branch tests on Windows. Two failures, which are repeats of longstanding trunk failures on Windows: http://www.zope.org/Collectors/Zope/1728
On Tue, 29 Mar 2005 18:12:51 -0500, Tim Peters <tim.peters@gmail.com> wrote:
[Tres]
- ZODBGuru uses 'svn rm' to zap the current ZODB on the trunk, replacing it with an 'svn:external' link to your ZODB 3.4 tag; and then tests it (could be an 'svn cp', but I don't see any benefit to maintaining a Zope-specific fork).
My problem. It will certainly be done via 9 "svn switch"s on my local machine first. As explained elsewhere, there are several other possible sources of integration problems here, and I can't know about those before actually trying it. It's possible, e.g., that I'll need to change ZODB 3.4 to worm around them, or switch ZODB to using a different ZConfig, etc. Can't predict here. [snip]
- ReleaseMaker uses 'svn cp' to create the release tag for Zope 2.8a2 (whatever it is being called).
Check.
When you use svn:external, it will be copied as is to the tag, so when the external files change, the files in the tag change as well. So a svn cp would be better I guess. Regards, Florian Schulze
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Florian Schulze wrote:
On Tue, 29 Mar 2005 18:12:51 -0500, Tim Peters <tim.peters@gmail.com> wrote:
[Tres]
- ZODBGuru uses 'svn rm' to zap the current ZODB on the trunk, replacing it with an 'svn:external' link to your ZODB 3.4 tag; and then tests it (could be an 'svn cp', but I don't see any benefit to maintaining a Zope-specific fork).
My problem. It will certainly be done via 9 "svn switch"s on my local machine first. As explained elsewhere, there are several other possible sources of integration problems here, and I can't know about those before actually trying it. It's possible, e.g., that I'll need to change ZODB 3.4 to worm around them, or switch ZODB to using a different ZConfig, etc. Can't predict here.
[snip]
- ReleaseMaker uses 'svn cp' to create the release tag for Zope 2.8a2 (whatever it is being called).
Check.
When you use svn:external, it will be copied as is to the tag, so when the external files change, the files in the tag change as well. So a svn cp would be better I guess.
I don't think so. The point is that each released version of Zope (already) depends on a specific, released version of ZODB; we don't *want* to be copying ZODB into the Zope tree in the repository, because that implicitly creates a fork (i.e., people check into the ZODB copy inside the Zope tree, which is Evil (TM).) When you use 'svn:externals', the referenced package itself is *not* part of the containing checkout; it is managed separately by the svn client (sort of like ESI and page fragments). Tim points out that there are a number of these external dependencies, including ZConfig and zdaemon, which are not directly part of ZODB either: it depends on them in the same way that Zope depends on ZODB. Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCSy2lGqWXf00rNCgRArecAJ9yZDl7hLH+cAO1eYeIePUB6JzbZQCeJjX7 wXM+tv0oKKpcDFpZd6CQPRI= =Cb8U -----END PGP SIGNATURE-----
Hello, Tres Seaver wrote:
When you use 'svn:externals', the referenced package itself is *not* part of the containing checkout; it is managed separately by the svn client (sort of like ESI and page fragments).
Yes this is true. But the differences between svn:externals and a copy are not so big. (the user nearly gets the same) With both you can get enough trouble. Even with svn:externals it's possible that someone checks something in to a tagged version. Subversion misses the feature to make tags read only and thats what you need in both cases.
Tim points out that there are a number of these external dependencies, including ZConfig and zdaemon, which are not directly part of ZODB either: it depends on them in the same way that Zope depends on ZODB.
My company uses subversion extensively, we have one rule: If we are in the same repository, we try to make a copy. If we have to different repositories and there are dependencies, we use svn:externals. Because there is one big Problem with svn:externals: If the repository is closed source and not available over Internet it is not possible to work at home with ssh checkouts. Hope this helps a little bit. bye by Wolfgang
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Wolfgang Langner wrote:
Hello,
Tres Seaver wrote:
When you use 'svn:externals', the referenced package itself is *not* part of the containing checkout; it is managed separately by the svn client (sort of like ESI and page fragments).
Yes this is true. But the differences between svn:externals and a copy are not so big. (the user nearly gets the same)
Copies are forks. We have a lot of experience over the past year with the pain those forks cause.
With both you can get enough trouble. Even with svn:externals it's possible that someone checks something in to a tagged version. Subversion misses the feature to make tags read only and thats what you need in both cases.
Note that the Zope SVN repository does not allow write access via 'svn:' URLs, so we can get the "read-only" effect by exploiting that. We should proabably also see what can be done to make commits into the 'tags' tree disallowed; I'm not sure if that is possible in a 'svn+ssh:' setup.
Tim points out that there are a number of these external dependencies, including ZConfig and zdaemon, which are not directly part of ZODB either: it depends on them in the same way that Zope depends on ZODB.
My company uses subversion extensively, we have one rule:
If we are in the same repository, we try to make a copy. If we have to different repositories and there are dependencies, we use svn:externals.
Because there is one big Problem with svn:externals:
If the repository is closed source and not available over Internet it is not possible to work at home with ssh checkouts.
Doesn't obtain here.
Hope this helps a little bit.
Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCTBKVGqWXf00rNCgRAioQAJ9gVzrxEkcqx8CsyOgN7+A21f/cfwCgouY4 hH3OLrh5fpQFCXq217DCWA0= =WekI -----END PGP SIGNATURE-----
[Florian Schulze]
When you use svn:external, it will be copied as is to the tag, so when the external files change, the files in the tag change as well. So a svn cp would be better I guess.
When I make an SVN tag, like repos/main/ZODB/tags/3.4.0a1, the iron intent is that no changes will ever be checked in on that tag. Otherwise the tag would be useless (a tag is meant to capture a frozen point in time). So there's no particular danger in specifying an honest-to-gosh tag in an svn:externals block, although SVN doesn't enforce read-only-ness of tags. If we're worried about that (I'm not), svn:externals also allows specifying a specific revision number to fetch. That should be bulletproof. OTOH, over the next few days, while Jim and I are trying to get other ZODB 3.4 changes made and merged in, it may actually be an advantage to point to ZODB trunk directories, so that ZODB changes show up in the Zope trunk at once too. That's half of how it works in Zope 2.7 (there, changes checked in from anywhere show up everywhere at once).
I'm merging ZODB 3.4 into Zope on a branch. As Tres noted earlier in a checkin comment, a test failure results, because at the ZODB sprint we fleshed out IDataManager and an older Zope3 class claiming to implement IDataManager no longer does. Since Zope trunk doesn't "own" either the IDataManager definition nor the old Zope3 code, it's unclear how best to proceed: ERROR: testInterface (zope.app.mail.tests.test_delivery.TestMailDataManager) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\Zope\lib\python\zope\app\mail\tests\test_delivery.py", line 46, in testInterface verifyObject(IDataManager, manager) File "C:\Code\Zope\lib\python\zope\interface\verify.py", line 93, in verifyObject return _verify(iface, candidate, tentative, vtype='o') File "C:\Code\Zope\lib\python\zope\interface\verify.py", line 60, in _verify raise BrokenImplementation(iface, n) BrokenImplementation: An object has failed to implement interface <InterfaceClass transaction.interfaces.IDataManager> The abort_sub attribute was not provided. An expedient hack would be to castrate ZODB 3.4's IDataManager definition, back to what it was before the ZODB sprint. That's unattractive for obvious reasons (like, e.g., that the old definition lied about what the interface actually is). Current Zope3 trunk will have the same problem with ZODB 3.4 in its MailDataManager class, so that's "the right place" to fix it. But then the repaired version also needs to be merged into the "old" Zope3 code Zope trunk is trying to use. "The old" zope/app/mail/delivery.py also uses the deprecated get_transaction(). Perhaps current Zope3 trunk's delivery.py could just be slammed into the tag used by (or a new tag created for) Zope trunk? get_transaction() is more troublesome than _just_ that, alas: there are about 160 instances of it across the stitched-in lib/python/zope, and Products/Five, code. This causes lots of new deprecation warnings when running the tests. These are easy to repair with 1-2 hours easy editing work, but again Zope trunk doesn't own the lib/python/zope code (where almost all of these appear).
Tim Peters wrote: [snip]
get_transaction() is more troublesome than _just_ that, alas: there are about 160 instances of it across the stitched-in lib/python/zope, and Products/Five, code. This causes lots of new deprecation warnings when running the tests. These are easy to repair with 1-2 hours easy editing work, but again Zope trunk doesn't own the lib/python/zope code (where almost all of these appear).
Right, lib/python/zope is actually Zope X3.0.0, and we didn't expect we'd need to *update* Zope X3.0 in order for it to work with Zope 2.8. The new ZODB version is having some repercussions there. Zope X3.0 was released against an older version of ZODB. I'm really at a loss at what to do there. I can spend time trying to shut up Zope X3 I guess, if that is the only option... What is the recipe of changing get_transaction(), is this documented somewhere? I don't think Five has much in the way of fundamental get_transaction(); there is one in browser.py and some in the test code. Regards, Martijn
Martijn Faassen wrote:
Tim Peters wrote: [snip]
get_transaction() is more troublesome than _just_ that, alas: there are about 160 instances of it across the stitched-in lib/python/zope, and Products/Five, code. This causes lots of new deprecation warnings when running the tests. These are easy to repair with 1-2 hours easy editing work, but again Zope trunk doesn't own the lib/python/zope code (where almost all of these appear).
Right, lib/python/zope is actually Zope X3.0.0, and we didn't expect we'd need to *update* Zope X3.0 in order for it to work with Zope 2.8. The new ZODB version is having some repercussions there. Zope X3.0 was released against an older version of ZODB. I'm really at a loss at what to do there.
Perhaps we should make a X3.0.1. This is fairly long overdue anyway. Alternatively, we could make a branch for use in 2.8. I don't think this would really be a problem.
I can spend time trying to shut up Zope X3 I guess, if that is the only option...
I think this is the best option.
What is the recipe of changing get_transaction(), is this documented somewhere?
It should be documented in the deprecation warning. Basically, rather than: get_transaction().commit() you should: import transaction ... transaction.commit() or from transaction import commit ... commit() Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
Martijn Faassen wrote:
Tim Peters wrote: [snip]
get_transaction() is more troublesome than _just_ that, alas: there are about 160 instances of it across the stitched-in lib/python/zope, and Products/Five, code. This causes lots of new deprecation warnings when running the tests. These are easy to repair with 1-2 hours easy editing work, but again Zope trunk doesn't own the lib/python/zope code (where almost all of these appear).
Right, lib/python/zope is actually Zope X3.0.0, and we didn't expect we'd need to *update* Zope X3.0 in order for it to work with Zope 2.8. The new ZODB version is having some repercussions there. Zope X3.0 was released against an older version of ZODB. I'm really at a loss at what to do there.
Perhaps we should make a X3.0.1. This is fairly long overdue anyway.
Alternatively, we could make a branch for use in 2.8. I don't think this would really be a problem.
We already have one: it was needed in order to remove the "excess" packages (the ones tagged as being in X3.0, without actually being installed by the zpkg stuff). $ cd lib/python/ $ svn propget svn:externals zope \ svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0-Zope-2.8-pr1/src/zope
I can spend time trying to shut up Zope X3 I guess, if that is the only option...
Adding the required methods to the zope.app.mail.delivery thingy should take less than an hour, I think. Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCTBPYGqWXf00rNCgRAnj4AJwLw1vxxRrqpS9T4EpZ3tVQL5L39ACfU/EX 6zFZCDkS5uh0raHfsXzsCqc= =rtqs -----END PGP SIGNATURE-----
Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jim Fulton wrote:
Martijn Faassen wrote: [snip]
Right, lib/python/zope is actually Zope X3.0.0, and we didn't expect we'd need to *update* Zope X3.0 in order for it to work with Zope 2.8. The new ZODB version is having some repercussions there. Zope X3.0 was released against an older version of ZODB. I'm really at a loss at what to do there.
Perhaps we should make a X3.0.1. This is fairly long overdue anyway.
Alternatively, we could make a branch for use in 2.8. I don't think this would really be a problem.
We already have one: it was needed in order to remove the "excess" packages (the ones tagged as being in X3.0, without actually being installed by the zpkg stuff).
You want to do the get_transaction() changes on that then, and not on the ZopeX3.0 branch proper?
I can spend time trying to shut up Zope X3 I guess, if that is the only option...
Adding the required methods to the zope.app.mail.delivery thingy should take less than an hour, I think.
I'm not following this. What required methods with what zope.app.mail.delivery? I thought we were talking about ZODB 3.4 related changes here. Regards, Martijn
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martijn Faassen wrote:
Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jim Fulton wrote:
Martijn Faassen wrote:
[snip]
Right, lib/python/zope is actually Zope X3.0.0, and we didn't expect we'd need to *update* Zope X3.0 in order for it to work with Zope 2.8. The new ZODB version is having some repercussions there. Zope X3.0 was released against an older version of ZODB. I'm really at a loss at what to do there.
Perhaps we should make a X3.0.1. This is fairly long overdue anyway.
Alternatively, we could make a branch for use in 2.8. I don't think this would really be a problem.
We already have one: it was needed in order to remove the "excess" packages (the ones tagged as being in X3.0, without actually being installed by the zpkg stuff).
You want to do the get_transaction() changes on that then, and not on the ZopeX3.0 branch proper?
Depends. If somebody has an appetite for making a 3.0.1 release (not me!) then make the branch compatible with ZODB 3.4. We can then re-branch to create the 2.8 extract (probably by merging the changes from our current branch).
I can spend time trying to shut up Zope X3 I guess, if that is the only option...
Adding the required methods to the zope.app.mail.delivery thingy should take less than an hour, I think.
I'm not following this. What required methods with what zope.app.mail.delivery? I thought we were talking about ZODB 3.4 related changes here.
ZODB 3.4 updates the transaction.IDataManager contract (documents what was always the real contract), which causes a test in that package to fail. Again, we can "fix" it on the 3.0 branch, if a release is planned there, or else just fix it on our own branch. Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCTB4iGqWXf00rNCgRAlSCAJ9DpgI3a0xMUR9XJK+dWmQb7eb2xQCfVDmY mIH6sohjv/H5c+j1hlD+fpg= =YZSP -----END PGP SIGNATURE-----
Zope trunk is using ZODB 3.4 now. It's been switched to use svn:externals to stitch in the 9 ZODB directories. For now they're pointing at ZODB trunk. A release tag will be made later (this is _not_ ready for release yet; now I can start doing the work I wanted to start doing on Monday <0.5 wink>). The Zope3 code is now gotten from new tag Zope3/tags/ZopeX3-3.0.0-Zope-2.8-a2. That code was purged of get_transaction() calls, and the IDataManager glitch got repaired there. If that code needs more changes, they should be made on Zope3/branches/ZopeX3-3.0.0-Zope-2.8, another tag made from the result, and the lib/python properties changed to point to that tag. Really helpful: you can do "svn propedit" followed by "svn up" to _temporarily_ redirect your local copy to a different source for svn:externals. If you "svn revert" the propedit changes before a commit, nobody else will be affected. A handful of get_transaction() calls were also removed from the Five code. Zope's setup.py got taught how to build ZODB 3.4's new IFBTrees. All tests should pass on Linux (the same two still fail on Windows). There should be no DeprecationWarnings related to transactions. I'm not sure svn will do a decent job of updating. We're simultaneously trying to delete Zope trunk's *copies* of ZODB code, and stitch the same things (well, directories of the same names) back in via svn:externals. At one point during the Zope/branches/tim-merge-zodb34 merge on my local box, svn got itself terminally confused, starting to create directories like lib/lib/python/ZODB (there are two "lib/"s in that -- not a typo), and griping endlessly about locks and non-existent files. No amount of "svn cleanup" could repair it, and I ended up checking out Zope trunk from scratch again. Then all the problems went away. If people have trouble updating on Linux, say so here, and we can share solutions. I hope that I hit problems just because I was doing the merge.
| At one point during the Zope/branches/tim-merge-zodb34 merge on my | local box, svn got itself terminally confused, starting to create | directories like lib/lib/python/ZODB (there are two "lib/"s in that -- | not a typo), and griping endlessly about locks and non-existent files. | No amount of "svn cleanup" could repair it, and I ended up checking | out Zope trunk from scratch again. Then all the problems went away. | | If people have trouble updating on Linux, say so here, and we can | share solutions. I hope that I hit problems just because I was doing | the merge. Got one problem so far: svn: Failed to add directory 'lib/python/BTrees/tests': object of the same name already exists Everything else seems to have updated with no problems to that point. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://www.enfoldsystems.com http://plone.org/about/team#dreamcatcher I have not yet begun to byte!
[Sidnei da Silva]
Got one problem so far:
svn: Failed to add directory 'lib/python/BTrees/tests': object of the same name already exists
Everything else seems to have updated with no problems to that point.
Ugh -- I'm afraid that's the very first of the 9 ZODB directories it tried to update, and svn just isn't smart enough to delete the existing directories before trying to fetch the new ones. Did you find a way to work around that (short of checking out Zope from scratch? the latter works fine, but is irritating)?
On Thu, Mar 31, 2005 at 03:33:11PM -0500, Tim Peters wrote: | Ugh -- I'm afraid that's the very first of the 9 ZODB directories it | tried to update, and svn just isn't smart enough to delete the | existing directories before trying to fetch the new ones. Indeed it was. I think its a Good Thing. Explicitness. | Did you find a way to work around that (short of checking out Zope | from scratch? the latter works fine, but is irritating)? Removing the only the complainers and svn up after that. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://www.enfoldsystems.com http://plone.org/about/team#dreamcatcher You can tune a piano, but you can't tuna fish. You can tune a filesystem, but you can't tuna fish. -- from the tunefs(8) man page
Jim Fulton wrote:
Martijn Faassen wrote:
Tim Peters wrote: [snip]
get_transaction() is more troublesome than _just_ that, alas: there are about 160 instances of it across the stitched-in lib/python/zope, and Products/Five, code. This causes lots of new deprecation warnings when running the tests. These are easy to repair with 1-2 hours easy editing work, but again Zope trunk doesn't own the lib/python/zope code (where almost all of these appear).
Right, lib/python/zope is actually Zope X3.0.0, and we didn't expect we'd need to *update* Zope X3.0 in order for it to work with Zope 2.8. The new ZODB version is having some repercussions there. Zope X3.0 was released against an older version of ZODB. I'm really at a loss at what to do there.
Perhaps we should make a X3.0.1. This is fairly long overdue anyway.
I'll see whether I can make some time to go through X3.0. in that case, to prepare for a X3.0.1. To do this work, I need ZODB 3.4 to be stitched into here, right: http://svn.zope.org/Zope3/branches/ZopeX3-3.0/ is there a resolution on how to do this yet?
Alternatively, we could make a branch for use in 2.8. I don't think this would really be a problem.
It's probably better to do it on X3.0 branch proper, and then extract a new 'more minimal' branch for Zope 2.8 (which we already have). It would be best if Tres did the latter, as he did it last time. :) We need to carefully check whether there aren't changes in ZopeX3.0 for Zope 2.8 that we need to port to the new branch. Perhaps Tres has an idea on how to approach all of this.
I can spend time trying to shut up Zope X3 I guess, if that
is the only option...
I think this is the best option.
What is the recipe of changing get_transaction(),
is this documented somewhere?
It should be documented in the deprecation warning.
Thanks; I haven't seen those warnings yet as I don't think ZODB 3.4 quite got merged into Zope 2.8 trunk yet. Regards, Martijn
[Martijn Faassen, on deprecation warnings]
Thanks; I haven't seen those warnings yet as I don't think ZODB 3.4 quite got merged into Zope 2.8 trunk yet.
Not even close yet. I'm doing the 3,4 integration work on Zope/branches/tim-merge-zodb34 and I'm not merging anything into the trunk until all the ZODB 3.4 integration issues are settled on that branch. Jim also wants to make changes to ZODB 3.4. And I need to make more changes to ZODB 3.4 too. In an ideal world, the five-integration branch would have been merged to the trunk during the PyCon week, and then all this later work could have started on Monday. As is, it didn't really start until last night. But from my POV, it's under control <wink>. I expect that the alpha2 release needs to slip until early next week, though.
[Tim Peters] ...
get_transaction() is more troublesome than _just_ that, alas: there are about 160 instances of it across the stitched-in lib/python/zope, and Products/Five, code. This causes lots of new deprecation warnings when running the tests. These are easy to repair with 1-2 hours easy editing work, but again Zope trunk doesn't own the lib/python/zope code (where almost all of these appear).
[Martijn Faassen]
Right, lib/python/zope is actually Zope X3.0.0, and we didn't expect we'd need to *update* Zope X3.0 in order for it to work with Zope 2.8.
I know. That's why I pushed and pushed to get the branch merged "early" -- I knew _something_ would go wrong, I just didn't know what <wink>. Jim and I knew about the ZODB-3.4-in-Zope-2.8 plans for quite a while, but I doubt that even Brian was aware of them. There's no way you could have known -- not your fault.
The new ZODB version is having some repercussions there. Zope X3.0 was released against an older version of ZODB. I'm really at a loss at what to do there. I can spend time trying to shut up Zope X3 I guess, if that is the only option... What is the recipe of changing get_transaction(), is this documented somewhere?
It is, but it would go faster if I did it. I rewrote all uses of get_transaction() in the Zope and Zope3 trunks on Monday, so I'm in practice. For the most part they amount to no more than what Jim said, but there are trickier cases than commit(). Suggestion: I make a new copy of Zope3/tags/ZopeX3-3.0.0-Zope-2.8-pr1/src/zope stitch that into Zope trunk (change the lib/python svn:externals to point to the new copy), do all the get_transaction() edits there, and repair the IDataManager glitch there too. This could easily be done before lunch today (my time <wink>). If people are agreeable, help me pick a name for the new copy; I have no idea what the "pr1" is supposed to mean in the current name, but would like to stick to whatever naming convention is in use there.
I don't think Five has much in the way of fundamental get_transaction(); ...
That's right -- it's almost all under lib/python/zope/. [Jim Fulton[
Perhaps we should make a X3.0.1. This is fairly long overdue anyway.
Alternatively, we could make a branch for use in 2.8. I don't think this would really be a problem.
Above, I'm volunteering to do the latter. Somehow I get the impression that sticking an unplanned X3.0.1 release on the critical path for Zope 2.8 wouldn't go over well here <wink>.
[Tim Peters] ...
Suggestion: I make a new copy of
Zope3/tags/ZopeX3-3.0.0-Zope-2.8-pr1/src/zope
stitch that into Zope trunk (change the lib/python svn:externals to point to the new copy), do all the get_transaction() edits there, and repair the IDataManager glitch there too. This could easily be done before lunch today (my time <wink>).
I assume nobody objects, since nobody objected <ahem>. I see that Zope3/branches/ZopeX3-3.0.0-Zope-2.8 already exists, has already made changes to Zope3 code, and that the Zope3/tags/ZopeX3-3.0.0-Zope-2.8-pr1 tag Zope trunk is currently using appears to be a tag made off of that branch. So I'll continue on that line: make edits in Zope3/branches/ZopeX3-3.0.0-Zope-2.8 and, when they're done, make a new tag from that branch and stitch the new tag into Zope trunk. If anyone objects, please do so ASAP.
Tim Peters wrote:
[Tim Peters] ...
Suggestion: I make a new copy of
Zope3/tags/ZopeX3-3.0.0-Zope-2.8-pr1/src/zope
stitch that into Zope trunk (change the lib/python svn:externals to point to the new copy), do all the get_transaction() edits there, and repair the IDataManager glitch there too. This could easily be done before lunch today (my time <wink>).
I assume nobody objects, since nobody objected <ahem>. I see that
Zope3/branches/ZopeX3-3.0.0-Zope-2.8
already exists, has already made changes to Zope3 code, and that the
Zope3/tags/ZopeX3-3.0.0-Zope-2.8-pr1
tag Zope trunk is currently using appears to be a tag made off of that branch. So I'll continue on that line: make edits in
Zope3/branches/ZopeX3-3.0.0-Zope-2.8
and, when they're done, make a new tag from that branch and stitch the new tag into Zope trunk.
If anyone objects, please do so ASAP.
This is exactly right! I should read all your mail, I'm typing at cross-purposes. :) Regards, Martijn
Tim Peters wrote:
[Tim Peters] ...
get_transaction() is more troublesome than _just_ that, alas: there are about 160 instances of it across the stitched-in lib/python/zope, and Products/Five, code. This causes lots of new deprecation warnings when running the tests. These are easy to repair with 1-2 hours easy editing work, but again Zope trunk doesn't own the lib/python/zope code (where almost all of these appear).
[Martijn Faassen]
Right, lib/python/zope is actually Zope X3.0.0, and we didn't expect we'd need to *update* Zope X3.0 in order for it to work with Zope 2.8.
I know. That's why I pushed and pushed to get the branch merged "early" -- I knew _something_ would go wrong, I just didn't know what <wink>.
But it isn't going wrong, as ZODB 3.4 is there yet, right? :) We only find out because you tried it on some other branch? I'm not clear on where this went on... [snip]
Suggestion: I make a new copy of
Zope3/tags/ZopeX3-3.0.0-Zope-2.8-pr1/src/zope
If we're going to go that route, no need to make a copy, just use the branch that's already there for exactly this purpose: http://svn.zope.org/Zope3/branches/ZopeX3-3.0.0-Zope-2.8/ and make a new -pr2 or -integration2 or whatever you like tag out of it when you're done.
stitch that into Zope trunk (change the lib/python svn:externals to point to the new copy), do all the get_transaction() edits there, and repair the IDataManager glitch there too. This could easily be done before lunch today (my time <wink>).
Just stitch back in the /branches/ZopeX3-3.0.0-Zope-2.8, and fix that, and then, when you're done, copy it to a new tag and update the external to that.
If people are agreeable, help me pick a name for the new copy; I have no idea what the "pr1" is supposed to mean in the current name, but would like to stick to whatever naming convention is in use there.
Right, pr1 is out of date. What about? Zope3/tags/ZopeX3-3.0.0-Zope-2.8-i2 for 'integration 2'? :) [snip]
Alternatively, we could make a branch for use in 2.8. I don't think this would really be a problem.
Above, I'm volunteering to do the latter. Somehow I get the impression that sticking an unplanned X3.0.1 release on the critical path for Zope 2.8 wouldn't go over well here <wink>.
Geez, that's true, thanks. :) I'm too addled today to even yammer about it a lot. Regards, Martijn
In this case, the answer is clear to me. :) We should fix the offending Z3 code to match the interface. Similarly, we should fix the uses of get_transaction. Jim Tim Peters wrote:
I'm merging ZODB 3.4 into Zope on a branch. As Tres noted earlier in a checkin comment, a test failure results, because at the ZODB sprint we fleshed out IDataManager and an older Zope3 class claiming to implement IDataManager no longer does. Since Zope trunk doesn't "own" either the IDataManager definition nor the old Zope3 code, it's unclear how best to proceed:
ERROR: testInterface (zope.app.mail.tests.test_delivery.TestMailDataManager) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\Zope\lib\python\zope\app\mail\tests\test_delivery.py", line 46, in testInterface verifyObject(IDataManager, manager) File "C:\Code\Zope\lib\python\zope\interface\verify.py", line 93, in verifyObject return _verify(iface, candidate, tentative, vtype='o') File "C:\Code\Zope\lib\python\zope\interface\verify.py", line 60, in _verify raise BrokenImplementation(iface, n) BrokenImplementation: An object has failed to implement interface <InterfaceClass transaction.interfaces.IDataManager>
The abort_sub attribute was not provided.
An expedient hack would be to castrate ZODB 3.4's IDataManager definition, back to what it was before the ZODB sprint. That's unattractive for obvious reasons (like, e.g., that the old definition lied about what the interface actually is).
Current Zope3 trunk will have the same problem with ZODB 3.4 in its MailDataManager class, so that's "the right place" to fix it. But then the repaired version also needs to be merged into the "old" Zope3 code Zope trunk is trying to use. "The old" zope/app/mail/delivery.py also uses the deprecated get_transaction(). Perhaps current Zope3 trunk's delivery.py could just be slammed into the tag used by (or a new tag created for) Zope trunk?
get_transaction() is more troublesome than _just_ that, alas: there are about 160 instances of it across the stitched-in lib/python/zope, and Products/Five, code. This causes lots of new deprecation warnings when running the tests. These are easy to repair with 1-2 hours easy editing work, but again Zope trunk doesn't own the lib/python/zope code (where almost all of these appear).
-- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
On Wed, 30 Mar 2005 17:59:49 -0500, Tim Peters <tim.peters@gmail.com> wrote:
[Florian Schulze]
When you use svn:external, it will be copied as is to the tag, so when the external files change, the files in the tag change as well. So a svn cp would be better I guess.
When I make an SVN tag, like repos/main/ZODB/tags/3.4.0a1, the iron intent is that no changes will ever be checked in on that tag. Otherwise the tag would be useless (a tag is meant to capture a frozen point in time). So there's no particular danger in specifying an honest-to-gosh tag in an svn:externals block, although SVN doesn't enforce read-only-ness of tags.
If we're worried about that (I'm not), svn:externals also allows specifying a specific revision number to fetch. That should be bulletproof.
OTOH, over the next few days, while Jim and I are trying to get other ZODB 3.4 changes made and merged in, it may actually be an advantage to point to ZODB trunk directories, so that ZODB changes show up in the Zope trunk at once too. That's half of how it works in Zope 2.7 (there, changes checked in from anywhere show up everywhere at once).
That was what I wanted to point out, you have to be sure that the svn:external in the tag is pointing to the specific tag in the ZODB trunk. I made that mistake once and justed wanted to warn about it. Regards, Florian Schulze
participants (9)
-
Andreas Jung -
Florian Schulze -
Jim Fulton -
Martijn Faassen -
Sidnei da Silva -
Tim Peters -
Tim Peters -
Tres Seaver -
Wolfgang Langner