Zope 2.12.9 broke ZMySQLDA in the middle of a stable release series
Hi, The latest Zope 2.12.9 release broke the last release of Products.ZMySQLDA. To demonstrate, bootstrap and run the attached buildout, then run: $ bin/py -c "import Shared.DC.ZRDB, Products.ZMySQLDA" You'll get the following traceback: Traceback (most recent call last): File "bin/py", line 107, in <module> exec _val File "<string>", line 1, in <module> File "/home/leo/opt/zmysqlda/eggs/Products.ZMySQLDA-3.1-py2.6.egg/Products/ZMySQLDA/__init__.py", line 90, in <module> import DA File "/home/leo/opt/zmysqlda/eggs/Products.ZMySQLDA-3.1-py2.6.egg/Products/ZMySQLDA/DA.py", line 243, in <module> os.path.join('Shared','DC','ZRDB','www','DBAdapterFolder_icon.gif'))} File "/home/leo/opt/zmysqlda/eggs/Zope2-2.12.9-py2.6-linux-x86_64.egg/App/ImageFile.py", line 77, in __init__ stat_info = os.stat(path) OSError: [Errno 2] No such file or directory: '/home/leo/opt/zmysqlda/eggs/Zope2-2.12.9-py2.6-linux-x86_64.egg/Shared/DC/ZRDB/www/DBAdapterFolder_icon.gif' Edit buildout.cfg and switch the extends directive to the commented out 2.12.8 tag and see that the above command shows no error, only a deprecation warning that is unrelated to the breakage above. Internal buildbots that tracked the 2.12 branch notified me of this problem, but in between catching up with all the traffic for the mailing lists and the Zope commit, and a day job, I didn't have enough time to track down the issue before the 2.12.9 release was made: there was less than 4 days between spinning off Products.ZSQLMethods and the new 2.12 release. The most trivial fix seems fairly obvious: ZMySQLDA should carry its own icon, but in any case, Zope 2.12 should not break compatibility in the middle of a stable series. I strongly suspect other Zope2 DB adapters to suffer from the same problem. I'm not sure what the proper fix should be. App.ImageFile.ImageFile could grow support for icons outside of software_home, or perhaps the spinning-off of Shared.DC.ZRDB should be delayed to the 2.13 series only, but in any case the 2.12 branch should be a safe upgrade for third party code as much as reasonably possible. suggestions? Cheers, Leo
Hi. On Tue, Jul 13, 2010 at 5:13 PM, Leonardo Rochael Almeida <leorochael@gmail.com> wrote:
The latest Zope 2.12.9 release broke the last release of Products.ZMySQLDA.
You are not by any chance interested in taking over maintenance of ZSQLMethods, are you?
You'll get the following traceback:
Traceback (most recent call last): File "bin/py", line 107, in <module> exec _val File "<string>", line 1, in <module> File "/home/leo/opt/zmysqlda/eggs/Products.ZMySQLDA-3.1-py2.6.egg/Products/ZMySQLDA/__init__.py", line 90, in <module> import DA File "/home/leo/opt/zmysqlda/eggs/Products.ZMySQLDA-3.1-py2.6.egg/Products/ZMySQLDA/DA.py", line 243, in <module> os.path.join('Shared','DC','ZRDB','www','DBAdapterFolder_icon.gif'))} File "/home/leo/opt/zmysqlda/eggs/Zope2-2.12.9-py2.6-linux-x86_64.egg/App/ImageFile.py", line 77, in __init__ stat_info = os.stat(path) OSError: [Errno 2] No such file or directory: '/home/leo/opt/zmysqlda/eggs/Zope2-2.12.9-py2.6-linux-x86_64.egg/Shared/DC/ZRDB/www/DBAdapterFolder_icon.gif'
The most trivial fix seems fairly obvious: ZMySQLDA should carry its own icon, but in any case, Zope 2.12 should not break compatibility in the middle of a stable series. I strongly suspect other Zope2 DB adapters to suffer from the same problem.
We documented it pretty clearly in the Zope 2.12.0 release notes, that you can no longer rely on software home or zope home. See for example http://docs.zope.org/zope2/releases/2.12/WHATSNEW.html#fully-eggified The code should have long been changed to import Shared.DC.ZRDB and then os.path.join(ZRDB.__file__, 'www', 'DBAdapterFolder_icon.gif') or better yet use its own icon. Both of these would have worked with all 2.12.x releases.
I'm not sure what the proper fix should be. App.ImageFile.ImageFile could grow support for icons outside of software_home, or perhaps the spinning-off of Shared.DC.ZRDB should be delayed to the 2.13 series only, but in any case the 2.12 branch should be a safe upgrade for third party code as much as reasonably possible.
We don't make guarantees on internals. Zope 2 always had a policy of allowing minor changes and new features to occur in the stable release series. It's not just pure bugfix releases. I think in this case the code in Products.ZMySQLDA should be changed to be compatible with Zope 2.12. Hanno
Am 13.07.2010, 17:44 Uhr, schrieb Hanno Schlichting <hanno@hannosch.eu>:
You are not by any chance interested in taking over maintenance of ZSQLMethods, are you?
As I've still got some stuff running on ZSQL and have previously championed it I'll also wave a hand, although I'm not that familiar with the internals so am not in a position to maintain it on my own. I'd like to see a scaled back version of ZSQL survive as I think there is a compelling TTW use case but I would drop the most of the dedicated dtml-sql methods as not suitable for this kind of use. Essentially I'd like to see templates generating statements with positional placeholders to be passed to .execute(stmt, paras) calls. I'm sure I remember seeing something like SmartTemplate a few years ago which looked suitable for this. Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226
Hi On Tue, Jul 13, 2010 at 12:44, Hanno Schlichting <hanno@hannosch.eu> wrote:
[...]
You'll get the following traceback:
Traceback (most recent call last): File "bin/py", line 107, in <module> exec _val File "<string>", line 1, in <module> File "/home/leo/opt/zmysqlda/eggs/Products.ZMySQLDA-3.1-py2.6.egg/Products/ZMySQLDA/__init__.py", line 90, in <module> import DA File "/home/leo/opt/zmysqlda/eggs/Products.ZMySQLDA-3.1-py2.6.egg/Products/ZMySQLDA/DA.py", line 243, in <module> os.path.join('Shared','DC','ZRDB','www','DBAdapterFolder_icon.gif'))} File "/home/leo/opt/zmysqlda/eggs/Zope2-2.12.9-py2.6-linux-x86_64.egg/App/ImageFile.py", line 77, in __init__ stat_info = os.stat(path) OSError: [Errno 2] No such file or directory: '/home/leo/opt/zmysqlda/eggs/Zope2-2.12.9-py2.6-linux-x86_64.egg/Shared/DC/ZRDB/www/DBAdapterFolder_icon.gif'
The most trivial fix seems fairly obvious: ZMySQLDA should carry its own icon, but in any case, Zope 2.12 should not break compatibility in the middle of a stable series. I strongly suspect other Zope2 DB adapters to suffer from the same problem.
We documented it pretty clearly in the Zope 2.12.0 release notes, that you can no longer rely on software home or zope home. See for example http://docs.zope.org/zope2/releases/2.12/WHATSNEW.html#fully-eggified
The code should have long been changed to import Shared.DC.ZRDB and then os.path.join(ZRDB.__file__, 'www', 'DBAdapterFolder_icon.gif') or better yet use its own icon. Both of these would have worked with all 2.12.x releases.
the call that failed is this: ImageFile(os.path.join('Shared','DC','ZRDB','www','DBAdapterFolder_icon.gif')) There is no explicit mention of "software home" anywhere, and there was no warning that this API was being used in a deprecated way. In fact, the use of software home happens inside ImageFile itself: def __init__(self, path, _prefix=None): import Globals # for data if _prefix is None: _prefix=getattr(getConfiguration(), 'softwarehome', PREFIX) elif type(_prefix) is not type(''): _prefix=package_home(_prefix) Of course I and most people on this list can read the ImageFile() call above and probably get a good hunch that sofware_home is being assumed somewhere, but a cursory reading of the call could also imply that a data file is simply being read relative to the Zope2 package location, and that this is supported behaviour of the ImageFile class.
I'm not sure what the proper fix should be. App.ImageFile.ImageFile could grow support for icons outside of software_home, or perhaps the spinning-off of Shared.DC.ZRDB should be delayed to the 2.13 series only, but in any case the 2.12 branch should be a safe upgrade for third party code as much as reasonably possible.
We don't make guarantees on internals. Zope 2 always had a policy of allowing minor changes and new features to occur in the stable release series. It's not just pure bugfix releases.
And I'm not disagreeing with the policy, but it can be argued that depending on the location of *data* files inside the Zope2 package is not necessarily relying on "guarantees on internals".
I think in this case the code in Products.ZMySQLDA should be changed to be compatible with Zope 2.12.
Obviously. Still, App.ImageFile.ImageFile (and any other Zope2 APIs that do rely on software_home) should give a clear warning when software_home is assumed, since it is the one assuming it. The exception raised by the ImageFile call now gives no clue on what actually went wrong and what a developer should do about it. Cheers, Leo
On Tue, Jul 13, 2010 at 7:05 PM, Leonardo Rochael Almeida <leorochael@gmail.com> wrote:
And I'm not disagreeing with the policy, but it can be argued that depending on the location of *data* files inside the Zope2 package is not necessarily relying on "guarantees on internals".
I'd call anything that relies on a specific file system layout to be internals. We only make guarantees on the semantics of the Python import system, not how packages and modules are stitched together in distributions and end up on the file system.
Obviously. Still, App.ImageFile.ImageFile (and any other Zope2 APIs that do rely on software_home) should give a clear warning when software_home is assumed, since it is the one assuming it. The exception raised by the ImageFile call now gives no clue on what actually went wrong and what a developer should do about it.
I agree with that. I thought the call to software home was being made inside ZMySQLDA and not in Zope 2 code. Hanno
On Tue, Jul 13, 2010 at 14:46, Hanno Schlichting <hanno@hannosch.eu> wrote:
On Tue, Jul 13, 2010 at 7:05 PM, Leonardo Rochael Almeida <leorochael@gmail.com> wrote:
And I'm not disagreeing with the policy, but it can be argued that depending on the location of *data* files inside the Zope2 package is not necessarily relying on "guarantees on internals".
I'd call anything that relies on a specific file system layout to be internals. We only make guarantees on the semantics of the Python import system, not how packages and modules are stitched together in distributions and end up on the file system.
Considering that the Zope source code has a dozen examples of App.ImageFile.ImageFile being used without a prefix (ex. "OFS.misc_"), I think third party developers should be excused to think they could follow the lead and expect their packages not to break during a stable series. I still think we should give them at least the 2.12 series to stop using ImageFile "incorrectly" without breaking their code, but I won't belabor the point any longer.
Obviously. Still, App.ImageFile.ImageFile (and any other Zope2 APIs that do rely on software_home) should give a clear warning when software_home is assumed, since it is the one assuming it. The exception raised by the ImageFile call now gives no clue on what actually went wrong and what a developer should do about it.
I agree with that. I thought the call to software home was being made inside ZMySQLDA and not in Zope 2 code.
The attached patch adds such a warning, and also reveals where Zope2 commits the same sins. If there are no objections I'll commit it to 2.12 and trunk. Finally, I'd like to ask that, when major changes land in a stable series (like the spinning off of a whole package), that we allow at least a week or two to pass before making a release, so people who have test runners for their apps running against a stable repository branch have time to adapt to the changes Cheers, Leo
On Wed, Jul 14, 2010 at 12:22 AM, Leonardo Rochael Almeida <leorochael@gmail.com> wrote:
The attached patch adds such a warning, and also reveals where Zope2 commits the same sins. If there are no objections I'll commit it to 2.12 and trunk.
Thanks for committing it.
Finally, I'd like to ask that, when major changes land in a stable series (like the spinning off of a whole package), that we allow at least a week or two to pass before making a release, so people who have test runners for their apps running against a stable repository branch have time to adapt to the changes
That sounds reasonable. I just wasn't aware that anyone except CMF actually tracks Zope 2 SVN and there would be any point in waiting at all. Unless the nightly CMF builds reported problems, I considered the code to be stable and ready for release. But to avoid any such issues, I'll restrain myself from doing any further feature changes to Zope 2.12. 2.13 is closing in fast enough and there's always a 2.14. Hanno
Am 15.07.2010, 10:35 Uhr, schrieb Hanno Schlichting <hanno@hannosch.eu>:
But to avoid any such issues, I'll restrain myself from doing any further feature changes to Zope 2.12. 2.13 is closing in fast enough and there's always a 2.14.
I'll check ZPsycopgDA and make sure we look at the mxODBCZope DA as well. I'm not sure what other Products are likely to be affected by these kind of changes. Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226
Am 15.07.2010, 13:39 Uhr, schrieb Charlie Clark <charlie.clark@clark-consulting.eu>:
I'll check ZPsycopgDA and make sure we look at the mxODBCZope DA as well. I'm not sure what other Products are likely to be affected by these kind of changes.
Taken a while but I can confirm that this also affects ZPsycopgDA. Furthermore, the term prefix is itself misleading as it can be either an absolute path or a package dictionary. Is there a preference? Shared.DC.ZRDB.__dict__ would seem to be the easiest. How are we to deal with existence connections which seem to ignore any changes? Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226
participants (3)
-
Charlie Clark -
Hanno Schlichting -
Leonardo Rochael Almeida