As the subject implies, I'm working on a package for Fedora. I've come across some missing dependencies, and some that are different versions. Because of the way that Python packages are put together (shared library environment), I'm trying to strip out everything that is not Zope specific (e.g. mechanize, docutils, etc.) that way Zope won't clobber already existing packages, and new packages won't clobber Zope. Here is my list (that I've discovered so far...) and some notes and questions. I would appreciate any information on resolving these. My list was taken from Zope-3.4.0c1/build/lib.linux-x86_64-2.5 (in my arch specific case...) after doing a `python install.py build` (side note, why is it called install.py and not setup.py?). BTrees -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope? ClientForm -- This currently exists in Fedora (version 0.2.7) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions? twisted -- This currently exists in Fedora (version 2.4.0) Diffing the directories reveals that is it mostly compatible, however, missing some features (I have a list). What parts are required by Zope? Some of the missing parts are no longer being maintained, or are incompatible with newest Twisted (2.5.0). Thoughts? mechanize -- This currently exists in Fedora (version 0.1.6) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions? persistent -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope? pytz -- This currently exists in Fedora (version 2006p) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions? The Zope versions do not have the .py extension. Possible problems with Zope if they do (i.e., lib resolving)? RestrictedPython -- Does not currently exist in Fedora as a separate package. Is this it? <http://pypi.python.org/pypi/RestrictedPython/3.4.2> Required version for Zope? What about newer versions? ThreadedAsync -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope? transaction -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If not Zope origin, is it useful outside of Zope? Also, there is an existing package in Fedora called python-zope-interface, version 3.0.1 I assume (in F8), and I'm wondering if it would be worth keeping separate (or at least available, updated, maybe with a Conflicts: zope) in Fedora and are there other parts of Zope (e.g. ZODB, ZEO) that would be useful by themselves? I'm sorry if this seems daunting (it is on my side ;) and/or demanding, but I would like this packaged the best possible way for both projects. Who knows, this might show up in future RHEL versions (maybe I'm delusional ;) Thank you for your time and patience. --Tim _________________________________________________________________________ / According to Kentucky state law, every person must take a bath at least \ \ once a year. / ------------------------------------------------------------------------- \ \ \ \ /\ ( ) .( o ).
Hi, speaking as the Zope 2 release manager: I am strongly opposed against splitting Zope yourself into different packages and modules. With Zope 2 depending from various Zope 3 packages (roughly 80-90) we have already the situation to keep track which packages belong together. This will become even more complicated when each Zope 3 package will have its own life-cycle. I doubt that you as a package can keep track in a reliable way with those requirements. It is somewhat hard for me to follow. So why can't you leave the Zope source packages as they are? Splitting up Zope into even more packages is even more error-prone. Re-packaged Zope distributions always raised more problems in the past than they really solved. The deployment for Zope-based applications is nowadays based on zc.buildout. Serious solutions providers never go with distribution-specific packages. Andreas --On 21. März 2008 11:57:31 -0700 Timothy Selivanow <timothy.selivanow@virtualxistenz.com> wrote:
As the subject implies, I'm working on a package for Fedora. I've come across some missing dependencies, and some that are different versions. Because of the way that Python packages are put together (shared library environment), I'm trying to strip out everything that is not Zope specific (e.g. mechanize, docutils, etc.) that way Zope won't clobber already existing packages, and new packages won't clobber Zope.
Here is my list (that I've discovered so far...) and some notes and questions. I would appreciate any information on resolving these. My list was taken from Zope-3.4.0c1/build/lib.linux-x86_64-2.5 (in my arch specific case...) after doing a `python install.py build` (side note, why is it called install.py and not setup.py?).
BTrees -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
ClientForm -- This currently exists in Fedora (version 0.2.7) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions?
twisted -- This currently exists in Fedora (version 2.4.0) Diffing the directories reveals that is it mostly compatible, however, missing some features (I have a list). What parts are required by Zope? Some of the missing parts are no longer being maintained, or are incompatible with newest Twisted (2.5.0). Thoughts?
mechanize -- This currently exists in Fedora (version 0.1.6) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions?
persistent -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
pytz -- This currently exists in Fedora (version 2006p) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions? The Zope versions do not have the .py extension. Possible problems with Zope if they do (i.e., lib resolving)?
RestrictedPython -- Does not currently exist in Fedora as a separate package. Is this it? <http://pypi.python.org/pypi/RestrictedPython/3.4.2> Required version for Zope? What about newer versions?
ThreadedAsync -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
transaction -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If not Zope origin, is it useful outside of Zope?
Also, there is an existing package in Fedora called python-zope-interface, version 3.0.1 I assume (in F8), and I'm wondering if it would be worth keeping separate (or at least available, updated, maybe with a Conflicts: zope) in Fedora and are there other parts of Zope (e.g. ZODB, ZEO) that would be useful by themselves?
I'm sorry if this seems daunting (it is on my side ;) and/or demanding, but I would like this packaged the best possible way for both projects. Who knows, this might show up in future RHEL versions (maybe I'm delusional ;)
Thank you for your time and patience.
--Tim ________________________________________________________________________ _ / According to Kentucky state law, every person must take a bath at least \ \ once a year. / ------------------------------------------------------------------------ - \ \ \ \ /\ ( ) .( o ).
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
-- ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: info@zopyx.com - Phone +49 - 7071 - 793376 Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535 Geschäftsführer/Gesellschafter: ZOPYX Limited, Birmingham, UK ------------------------------------------------------------------------ E-Publishing, Python, Zope & Plone development, Consulting
On Fri, 2008-03-21 at 20:09 +0100, Andreas Jung wrote:
Hi,
speaking as the Zope 2 release manager: I am strongly opposed against splitting Zope yourself into different packages and modules. With Zope 2 depending from various Zope 3 packages (roughly 80-90) we have already the situation to keep track which packages belong together.
My concern is more with striping out the non-Zope pieces. If there are pieces of Zope that are useful outside of Zope (e.g. in some other project, or someones personal code...) I thought it would be nice to offer those as a separate piece so that someone could use just that piece. Having 80-90 different RPMs _would_ be rather unmaintainable. I'd only pull out the first few useful ones, and then others by-demand. My biggest concern is removing the non-Zope parts. As far as Zope2 is concerned, I'll cross that bridge later. Newest technology is /generally/ the focus of Fedora.
This will become even more complicated when each Zope 3 package will have its own life-cycle. I doubt that you as a package can keep track in a reliable way with those requirements. It is somewhat hard for me to follow.
The Eclipse project has done an awesome job, and so has Fedora in keeping up with it (they actually work very closely together). I'm not concerned with ripping Zope /completely/ apart, but rather I thought /some/ parts might be useful by themselves, like the ZODB for example.
So why can't you leave the Zope source packages as they are? Splitting up Zope into even more packages is even more error-prone. Re-packaged Zope distributions always raised more problems in the past than they really solved. The deployment for Zope-based applications is nowadays based on zc.buildout.
So, are you saying that you would rather Zope not be in Fedora, or any other distro, and just leave it up to the user?
Serious solutions providers never go with distribution-specific packages.
Some do, but that's not the focus of me packaging Zope. Since Fedora's focus is not for production (yes, people do use Fedora in production, e.g. NASA, but that is their onus), but rather development and blazing new trails. This would provide more code and utilities for people to use and experiment with. --Tim __________________________________________________________ / As of next Tuesday, C will be flushed in favor of COBOL. \ \ Please update your programs. / ---------------------------------------------------------- \ \ \ \ /\ ( ) .( o ).
--On 21. März 2008 13:39:08 -0700 Timothy Selivanow <timothy.selivanow@virtualxistenz.com> wrote:
On Fri, 2008-03-21 at 20:09 +0100, Andreas Jung wrote:
Hi,
speaking as the Zope 2 release manager: I am strongly opposed against splitting Zope yourself into different packages and modules. With Zope 2 depending from various Zope 3 packages (roughly 80-90) we have already the situation to keep track which packages belong together.
My concern is more with striping out the non-Zope pieces. If there are pieces of Zope that are useful outside of Zope (e.g. in some other project, or someones personal code...) I thought it would be nice to offer those as a separate piece so that someone could use just that piece. Having 80-90 different RPMs _would_ be rather unmaintainable. I'd only pull out the first few useful ones, and then others by-demand. My biggest concern is removing the non-Zope parts.
In general we want to get rid of 3rd-party packages we don't want to maintain on our own. Right now we have several 3rd-party packages like Docutils in our svn for several reasons. Some of those packages are basically frozen (because we made local changes e.g. for security reasons). So if you rip out those packages you have to guarantee that they will _never_ be updated by a new official package (because local changes might get lost). On the other hand you must ensure that changes to our local 3rd-party packages will be available through your packaging mechanism. As said: this is a challenging task for us right now - it will be even more challenging and more error-prone for outsiders.
As far as Zope2 is concerned, I'll cross that bridge later. Newest technology is /generally/ the focus of Fedora.
This will become even more complicated when each Zope 3 package will have its own life-cycle. I doubt that you as a package can keep track in a reliable way with those requirements. It is somewhat hard for me to follow.
The Eclipse project has done an awesome job, and so has Fedora in keeping up with it (they actually work very closely together). I'm not concerned with ripping Zope /completely/ apart, but rather I thought /some/ parts might be useful by themselves, like the ZODB for example.
Since Zope (2+3) are officially only blessed for Python 2.4 you'll have problems if you're going with the newest Python 2.5 or even 2.6 version. The trend in all Zope-related projects (Zope 2,3, Grok, Plone) is definitely: self-contained installation. Why self-contained? Different projects require different modules in different versions. Messing a Python installation with e.g. ZODB from different versions will end up in an disaster. So the message of the Zope community is in general: keep your Python installation clean and use zc.buildout or virtualenv for installing your other modules within an isolated environment. This is meanwhile considered best-practice. Using Python eggs and easy_install you have _the official_ tools in your hand for installing 3rd-party packages..that's the Pythonic way to do it, not using some distro packaging tool.
So why can't you leave the Zope source packages as they are? Splitting up Zope into even more packages is even more error-prone. Re-packaged Zope distributions always raised more problems in the past than they really solved. The deployment for Zope-based applications is nowadays based on zc.buildout.
So, are you saying that you would rather Zope not be in Fedora, or any other distro, and just leave it up to the user?
This was never my point. For me it would make more sense that the packages would take the official release and re-package it as a whole without splitting it up. Dependency management should happen on our side as a whole, not on the distribution site. I don't know how important Zope distro packages are for users. My personal impression is: not so important. Zope packages at this point might be good for getting into touch with Zope but for deployment packages are basically no used - but correct me if I should be wrong. Since a while we have tools in place to bootstrap Zope 2/3 projects, Plone installations and Grok installations very easily and in a reliable way. I know that those approaches don't comply directly with your package mechanisms but I would expect that the "offical" installation and deployment stories by the Zope community should be taken into account (in some way).
Serious solutions providers never go with distribution-specific packages.
Some do, but that's not the focus of me packaging Zope. Since Fedora's focus is not for production (yes, people do use Fedora in production, e.g. NASA, but that is their onus), but rather development and blazing new trails. This would provide more code and utilities for people to use and experiment with.
See above. Distro specific package are perhaps another change for gettting into touch with Zope but I am not sure how important that is. I can imagine that people hear of Zope technology and then visit the related webpage and check the installation notes (which are basically not specific to a dstribution). One last thought: you might check out the packaging efforts by the Plone project. They provide possibly also packages for Fedora on their own (which includes Zope 2 afaik (the "universal installer")). It might make sense to share experiences with them and leverage from their experiences. But don't get me wrong. I have nothing against distro-specific packages but they should be done in the right way. "Right" means for the user: they are consistent with the official documentation and best-practices as used within the offical Zope world - "Right" for the supporters within the Zope world: we don't have to deal with issues caused distro-specific packages (e.g. configurations are different from the standard, paths are different from the standard etc.). Andreas
On Sat, 2008-03-22 at 08:05 +0100, Andreas Jung wrote:
In general we want to get rid of 3rd-party packages we don't want to maintain on our own.
That's perfectly sane :) There is no sustainable reason to lug around some other project's code in yours in the Open Source world.
Right now we have several 3rd-party packages like Docutils in our svn for several reasons. Some of those packages are basically frozen (because we made local changes e.g. for security reasons). So if you rip out those packages you have to guarantee that they will _never_ be updated by a new official package (because local changes might get lost). On the other hand you must ensure that changes to our local 3rd-party packages will be available through your packaging mechanism. As said: this is a challenging task for us right now - it will be even more challenging and more error-prone for outsiders.
Now when I say "rip out", I don't mean repackage (make a sub RPM), I mean remove from the RPM that I am making. I don't want to provide a "new" Docutils. I'm actually trying to place zope in the standard (for Redhat) python site-packages location while guaranteeing that the zope RPM doesn't provide more than it needs, and doesn't clobber other packages, like the native Docutils. If I knew what versions the 3rd party pieces were, I could make a diff (or the Zope project could provide one) and I could get those into the official Fedora RPMs (esp. if they are security fixes). Fedora doesn't like keeping those patches forever, so I would also get those pushed upstream.
The trend in all Zope-related projects (Zope 2,3, Grok, Plone) is definitely: self-contained installation. Why self-contained? Different projects require different modules in different versions.
That is completely understandable from a Service Provider standpoint, but not a distro. However, coming from a Service Provider background (my day job, as it were), I definitely like reproducibility, and a distro package (i.e. RPM) is a good place to start. It's very annoying and time consuming to replicate by hand the environment that is going to appear on several servers, esp. over long periods of time (not to mention upgrades/security, etc...)
Messing a Python installation with e.g. ZODB from different versions will end up in an disaster. So the message of the Zope community is in general: keep your Python installation clean and use zc.buildout or virtualenv for installing your other modules within an isolated environment. This is meanwhile considered best-practice. Using Python eggs and easy_install you have _the official_ tools in your hand for installing 3rd-party packages..that's the Pythonic way to do it, not using some distro packaging tool.
One could argue that the distro over-rides all. In a distro role, your job is to provide the least common denominator, not cater to every crazy developer and their environment. They should be more than capable of maintaining that themselves (install several different versions of python, zope, et al. in their home dir if need be).
This was never my point. For me it would make more sense that the packages would take the official release and re-package it as a whole without splitting it up. Dependency management should happen on our side as a whole, not on the distribution site.
Yes, but I still have to meet those dependencies. I was just hoping that there would be pieces (sounds like ZODB and zope.publisher would be good candidates from some of the other threads) that would be useful outside of the whole Zope Application Server scope. I assumed that as a (open source) software project, Zope would be delighted that people were using their code in new and creative ways. I wanted to help facilitate that by offering those pieces as a stand-alone library.
I don't know how important Zope distro packages are for users. My personal impression is: not so important.
Developers, yes, not very important. /Users/ on the other hand are a completely different animal, and packages are _very_ important. I suppose it's a matter of semantics on what exactly a user is in which scope.
Zope packages at this point might be good for getting into touch with Zope but for deployment packages are basically no used - but correct me if I should be wrong. Since a while we have tools in place to bootstrap Zope 2/3 projects, Plone installations and Grok installations very easily and in a reliable way.
It matters in an unattended, (e.g.) 100+ server enviro where reproducibility is a must.
I know that those approaches don't comply directly with your package mechanisms but I would expect that the "offical" installation and deployment stories by the Zope community should be taken into account (in some way).
Absolutely. Those are important. I would be delighted to read some of those if someone knew/had some.
One last thought: you might check out the packaging efforts by the Plone project. They provide possibly also packages for Fedora on their own (which includes Zope 2 afaik (the "universal installer")). It might make sense to share experiences with them and leverage from their experiences.
Sounds like they will be the next list I hit up (sigh...my number of subscribed lists is exponentially increasing ;) I've been kinda following the egg-ification and zope.producer dependency reduction threads, and it seems like all of that would be good for me too (any packager really...)
But don't get me wrong. I have nothing against distro-specific packages but they should be done in the right way. "Right" means for the user: they are consistent with the official documentation and best-practices as used within the offical Zope world - "Right" for the supporters within the Zope world: we don't have to deal with issues caused distro-specific packages (e.g. configurations are different from the standard, paths are different from the standard etc.).
Which is why I'm here asking for input. Perhaps I didn't start out correctly from a Zope'ish POV, but I needed to know where the conflicts were going to show up. If I packaged Zope as-is, placing it in (what I would think as) the standard python site-packages, it would never pass scrutiny because of the extra stuff it has, and would never be included in the distro. In older packages that /were/ in Fedora (they are no longer...), Zope was placed in a completely isolated environment, making it *only* useful as an web application server. I would rather have a package that upstream approves of and participates in, than blindly create it in a silo not caring. It never turns out well in the long run with the latter. --Tim __________________________________________________________________ / I saw a subliminal advertising executive, but only for a second. \ \ -- Steven Wright / ------------------------------------------------------------------ \ \ \ \ /\ ( ) .( o ).
On Mar 26, 2008, at 01:12 , Timothy Selivanow wrote:
I would rather have a package that upstream approves of and participates in, than blindly create it in a silo not caring. It never turns out well in the long run with the latter.
You may be fighting windmills here. Previous attempts at creating packaged versions, like RPMs, have flopped badly. On the one hand they tend to be behind the current release quickly. On the other hand they tend to introduce their own idea of how things should be laid out, which invariably clashes with what the developers/ supporters use. They tend to introduce scripts and scaffolding that do not exist in the standard tarball, which will lead to blank stares, such as start/stop scripts. Also, the goals that are cited for a packaged version, such as reproduceability of a given setup, are reached by the other installation mechanisms that are used as well, be it manually building a tarball or using helpers like zc.buildout. What it all boils down to (and has boiled down to in the past) is that people will be told to abandon the prepackaged version as soon as they need any kind of support on the "official" zope mailing lists. The notion that you build Zope (and in most cases Python as well) yourself instead of using anything pre-packaged is so deeply ingrained that no one has ever created a successful distribution-supplied package. Don't get me wrong, I'm not making any kind of value judgment, I'm just telling you what to expect. You may already be in the situation you are trying to avoid because the upstream may never approve of and support what you're attempting to do. jens
Timothy Selivanow wrote at 2008-3-25 17:12 -0700:
... Now when I say "rip out", I don't mean repackage (make a sub RPM), I mean remove from the RPM that I am making. I don't want to provide a "new" Docutils.
That Zope ships with its own "Docutils" comes from the fact that the standard one has a big security hole. Security issues are harmless for normal "Docutils" use but are critical when they can be exploited from a hostile internet.... -- Dieter
On Wed, Mar 26, 2008 at 09:20:27PM +0100, Dieter Maurer wrote:
Timothy Selivanow wrote at 2008-3-25 17:12 -0700:
... Now when I say "rip out", I don't mean repackage (make a sub RPM), I mean remove from the RPM that I am making. I don't want to provide a "new" Docutils.
That Zope ships with its own "Docutils" comes from the fact that the standard one has a big security hole.
Which one? The one that lets you embed any file on the filesystem into a web page? http://docutils.sourceforge.net/docs/howto/security.html I didn't know Zope's bundled version of docutils fixed that. In any case, the src/docutils in the Zope 3.2 tree either doesn't have the fix, or it doesn't work. I tested it and ended up closing that hole in an application myself. Marius Gedminas -- Alan Turing thought about criteria to settle the question of whether machines can think, a question of which we now know that it is about as relevant as the question of whether submarines can swim. -- Dijkstra
--On 27. März 2008 20:42:50 +0200 Marius Gedminas <mgedmin@b4net.lt> wrote:
On Wed, Mar 26, 2008 at 09:20:27PM +0100, Dieter Maurer wrote:
Timothy Selivanow wrote at 2008-3-25 17:12 -0700:
... Now when I say "rip out", I don't mean repackage (make a sub RPM), I mean remove from the RPM that I am making. I don't want to provide a "new" Docutils.
That Zope ships with its own "Docutils" comes from the fact that the standard one has a big security hole.
Which one? The one that lets you embed any file on the filesystem into a web page?
http://docutils.sourceforge.net/docs/howto/security.html
I didn't know Zope's bundled version of docutils fixed that. In any case, the src/docutils in the Zope 3.2 tree either doesn't have the fix, or it doesn't work. I tested it and ended up closing that hole in an application myself.
At least Zope 2 uses Docutils with the related options disabled. No idea about Zope 3.2. -aj
Disclaimer: my knowledge is not complete, and my track record of writing emails late at night is not very good. Take this with a big grain of salt. On Fri, Mar 21, 2008 at 11:57:31AM -0700, Timothy Selivanow wrote:
As the subject implies, I'm working on a package for Fedora. I've come across some missing dependencies, and some that are different versions. Because of the way that Python packages are put together (shared library environment), I'm trying to strip out everything that is not Zope specific (e.g. mechanize, docutils, etc.) that way Zope won't clobber already existing packages, and new packages won't clobber Zope.
Be careful: I wouldn't be surprised if the foreign packages in the Zope tree have bugfixes applied to them. I trust those fixes were also sent to their respective upstreams as well, but it may make it a bit more difficult to find out which upstream version is really needed.
Here is my list (that I've discovered so far...) and some notes and questions. I would appreciate any information on resolving these. My list was taken from Zope-3.4.0c1/build/lib.linux-x86_64-2.5 (in my arch specific case...) after doing a `python install.py build` (side note, why is it called install.py and not setup.py?).
Good question. It's called setup.py in the Subversion tree -- but the tarball is not based on the old monolithic SVN tree any more.
BTrees -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
It is part of ZODB. It may be useful outside of Zope.
persistent -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
ZODB again.
transaction -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If not Zope origin, is it useful outside of Zope?
ZODB again.
ThreadedAsync -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
ZODB again.
ClientForm -- This currently exists in Fedora (version 0.2.7) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions?
mechanize -- This currently exists in Fedora (version 0.1.6) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions?
I'm pretty sure these two are only used for zope.testbrowser. It ought to be sufficient to take the latest upstream version and see if zope.testbrowser's test suite passes with it. Benji York is the one with most knowledge about zope.testbrowser and its dependencies. If he offers different advice, listen to him, not to me.
twisted -- This currently exists in Fedora (version 2.4.0) Diffing the directories reveals that is it mostly compatible, however, missing some features (I have a list). What parts are required by Zope? Some of the missing parts are no longer being maintained, or are incompatible with newest Twisted (2.5.0). Thoughts?
AFAIK twisted is only bundled with Zope 3 for convenience. It is used for its HTTP server as one of the alternatives (the other ones being ZServer and WSGI), but even that part is not used by default.
pytz -- This currently exists in Fedora (version 2006p) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions? The Zope versions do not have the .py extension. Possible problems with Zope if they do (i.e., lib resolving)?
I think any non-broken pytz version should work. (Ubuntu once shipped a completely broken one.)
RestrictedPython -- Does not currently exist in Fedora as a separate package. Is this it? <http://pypi.python.org/pypi/RestrictedPython/3.4.2> Required version for Zope? What about newer versions?
I honestly have no idea.
Also, there is an existing package in Fedora called python-zope-interface, version 3.0.1 I assume (in F8), and I'm wondering if it would be worth keeping separate (or at least available, updated, maybe with a Conflicts: zope) in Fedora and are there other parts of Zope (e.g. ZODB, ZEO) that would be useful by themselves?
Yes and no. The Zope 3 upstream is now split into hundreds of little packages (Python eggs), each registered on the Python Package Index, with intricate and sometimes suboptimal dependency lists. If there's an easy way to convert a Python egg to an RPM, you may want to look at this, but beware: the dust hasn't fully settled yet. There's no guarantee that the packages on PyPI will work with each other at any given point of time. The Zope 3 Known Good Set fixes that, so you may want to look into it: http://wiki.zope.org/zope3/FAQGeneral#what-is-the-kgs-known-good-set On the other hand having multiple small packages will make your job harder.
I'm sorry if this seems daunting (it is on my side ;) and/or demanding, but I would like this packaged the best possible way for both projects. Who knows, this might show up in future RHEL versions (maybe I'm delusional ;)
The current consensus in the Zope community seems to be "avoid distribution packages, build your own Python from sources, then install Zope from sources in a sandbox". My personal experience with Zope 2 Debian packages reinforces the notion, although I still stubbornly stick with Python that comes with my distro. Well, sudo apt-get install zope2.9-sandbox was *very* nice for playing around in a sandbox, but for production use I will install Zope from sources. And I will probably have multiple different installations of different versions of Zope in different sandboxes, because each upgrade has to be carefully tested and scheduled with the clients. That's the voice of practicality. The voice of idealism says that yes, every little Zope package ought to be packaged in each popular distro, and these should work together nicely, and be bug-free and fully supported. It will take a lot of time and work to reach that state, and I'm afraid most of the Zopers are too busy building production applications to be much help with this. At least we have automated test suites...
Thank you for your time and patience.
Marius Gedminas -- Despite all appearances, your boss is a thinking, feeling, human being.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Marius Gedminas wrote:
Disclaimer: my knowledge is not complete, and my track record of writing emails late at night is not very good. Take this with a big grain of salt.
On Fri, Mar 21, 2008 at 11:57:31AM -0700, Timothy Selivanow wrote:
As the subject implies, I'm working on a package for Fedora. I've come across some missing dependencies, and some that are different versions. Because of the way that Python packages are put together (shared library environment), I'm trying to strip out everything that is not Zope specific (e.g. mechanize, docutils, etc.) that way Zope won't clobber already existing packages, and new packages won't clobber Zope.
I would also emphasize (maybe the OP knows this) that it isn't going to be sensible to rely on the Zope3 packages as canonical for packaging "Zope": "Zope3" is not an upgrade / replacement for "Zope2". In particular, Zope2 depends on a specific subset of the Zope3 packages, but also includes a bunch of other packages. The OP might find the following source distributions instructive: - http://dist.repoze.org/zopelib-2.10.5.0.tar.gz contains the Zope2 packages plus the relevant Zope3 code, drawn from Zope 3.3.2. It also includes most of the "foreign" packages the OP identified, except those in the ZODB egg. - http://dist.repoze.org/ZODB3-3.7.2.tar.gz contains the ZODB packages, including transaction and ThreadedAsync. This is the version that the zopelib distribution depends on.
Be careful: I wouldn't be surprised if the foreign packages in the Zope tree have bugfixes applied to them. I trust those fixes were also sent to their respective upstreams as well, but it may make it a bit more difficult to find out which upstream version is really needed.
Here is my list (that I've discovered so far...) and some notes and questions. I would appreciate any information on resolving these. My list was taken from Zope-3.4.0c1/build/lib.linux-x86_64-2.5 (in my arch specific case...) after doing a `python install.py build` (side note, why is it called install.py and not setup.py?).
Good question. It's called setup.py in the Subversion tree -- but the tarball is not based on the old monolithic SVN tree any more.
The Zope3 tarball is already EOLed (there won't be a 3.5 version released as a monolithic tarball, AFAIK). The Zope2 tarball will undergo one final release series (2.11.x), with plans / discussion / work underway to decompose it as eggs for the 2.12 release.
BTrees -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
It is part of ZODB. It may be useful outside of Zope.
persistent -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
ZODB again.
transaction -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If not Zope origin, is it useful outside of Zope?
ZODB again.
Now split out from th ZODB trunk (maybe not released as such yet).
ThreadedAsync -- Does not currently exist in Fedora as a separate package. Is this of Zope origin? (If not, where? Required version?) If Zope origin, is it useful outside of Zope?
ZODB again.
Gone in the ZODB trunk (again, maybe not yet released).
ClientForm -- This currently exists in Fedora (version 0.2.7) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions?
mechanize -- This currently exists in Fedora (version 0.1.6) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions?
I'm pretty sure these two are only used for zope.testbrowser. It ought to be sufficient to take the latest upstream version and see if zope.testbrowser's test suite passes with it.
Benji York is the one with most knowledge about zope.testbrowser and its dependencies. If he offers different advice, listen to him, not to me.
twisted -- This currently exists in Fedora (version 2.4.0) Diffing the directories reveals that is it mostly compatible, however, missing some features (I have a list). What parts are required by Zope? Some of the missing parts are no longer being maintained, or are incompatible with newest Twisted (2.5.0). Thoughts?
AFAIK twisted is only bundled with Zope 3 for convenience. It is used for its HTTP server as one of the alternatives (the other ones being ZServer and WSGI), but even that part is not used by default.
pytz -- This currently exists in Fedora (version 2006p) Diffing the directories and files reveals differences. What version is required for Zope? What about newer versions? The Zope versions do not have the .py extension. Possible problems with Zope if they do (i.e., lib resolving)?
I think any non-broken pytz version should work. (Ubuntu once shipped a completely broken one.)
RestrictedPython -- Does not currently exist in Fedora as a separate package. Is this it? <http://pypi.python.org/pypi/RestrictedPython/3.4.2> Required version for Zope? What about newer versions?
I honestly have no idea.
Also, there is an existing package in Fedora called python-zope-interface, version 3.0.1 I assume (in F8), and I'm wondering if it would be worth keeping separate (or at least available, updated, maybe with a Conflicts: zope) in Fedora and are there other parts of Zope (e.g. ZODB, ZEO) that would be useful by themselves?
Yes and no.
The Zope 3 upstream is now split into hundreds of little packages (Python eggs), each registered on the Python Package Index, with intricate and sometimes suboptimal dependency lists. If there's an easy way to convert a Python egg to an RPM, you may want to look at this, but beware: the dust hasn't fully settled yet. There's no guarantee that the packages on PyPI will work with each other at any given point of time. The Zope 3 Known Good Set fixes that, so you may want to look into it: http://wiki.zope.org/zope3/FAQGeneral#what-is-the-kgs-known-good-set
On the other hand having multiple small packages will make your job harder.
I'm sorry if this seems daunting (it is on my side ;) and/or demanding, but I would like this packaged the best possible way for both projects. Who knows, this might show up in future RHEL versions (maybe I'm delusional ;)
The current consensus in the Zope community seems to be "avoid distribution packages, build your own Python from sources, then install Zope from sources in a sandbox". My personal experience with Zope 2 Debian packages reinforces the notion, although I still stubbornly stick with Python that comes with my distro.
The day you spend four hours chasing down bugs caused by a distribution-shipped add-on, either added to or removed from /usr/lib/python2.4/site-packages, will be the day you swear off of it. Maybe if you've started using 'virtualenv --no-site-packages' you can defer that day indefinitely. If I were trying to do the job the OP is tackling, I would put all the Zope packages somewhere like '/usr/share/zope2/lib/python2.4', and arrange for the Zopish scripts to put that directory on the site path, *omitting* /usr/lib/python2.4/site-packages (perhaps via a custom site.py). I don't think I would relish trying to get Zope to work atop a random set of distro-managed packages.
Well, sudo apt-get install zope2.9-sandbox was *very* nice for playing around in a sandbox, but for production use I will install Zope from sources. And I will probably have multiple different installations of different versions of Zope in different sandboxes, because each upgrade has to be carefully tested and scheduled with the clients.
That's the voice of practicality.
The voice of idealism says that yes, every little Zope package ought to be packaged in each popular distro, and these should work together nicely, and be bug-free and fully supported. It will take a lot of time and work to reach that state, and I'm afraid most of the Zopers are too busy building production applications to be much help with this.
Yup, exactly.
At least we have automated test suites...
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH5FoJ+gerLs4ltQ4RAlMXAKC9fuQv0/sn/ludEzyi+hVs/FuouACgk1s6 3p5mllkmk3L6OgUXrTNqB2Q= =VBTU -----END PGP SIGNATURE-----
On Mar 21, 2008, at 8:59 PM, Tres Seaver wrote:
but also includes a bunch of other packages. The OP might find the following source distributions instructive:
- http://dist.repoze.org/zopelib-2.10.5.0.tar.gz contains the Zope2 packages plus the relevant Zope3 code, drawn from Zope 3.3.2. It also includes most of the "foreign" packages the OP identified, except those in the ZODB egg.
Actually this distro *also* includes the ZODB packages. I wasn't able to excise them.
- http://dist.repoze.org/ZODB3-3.7.2.tar.gz contains the ZODB packages, including transaction and ThreadedAsync. This is the version that the zopelib distribution depends on.
zopelib doesn't depend on this package, but repoze.zope2 does (just to get the most recent ZODB release; the installation ordering ensures that this one is installed last, and so is hopefully found first).
The voice of idealism says that yes, every little Zope package ought to be packaged in each popular distro, and these should work together nicely, and be bug-free and fully supported. It will take a lot of time and work to reach that state, and I'm afraid most of the Zopers are too busy building production applications to be much help with this.
Yup, exactly.
+1 on fewer packages, particularly for Zope2, which is a lovable twisted mess of subtle interdependencies between packages that mostly have no lives on their own anyway. - C
participants (7)
-
Andreas Jung -
Chris McDonough -
Dieter Maurer -
Jens Vagelpohl -
Marius Gedminas -
Timothy Selivanow -
Tres Seaver