Re: [Zope-dev] [Zope3-checkins] [Checkins] SVN: zope.app.component/trunk/setup.py set missing minimum version
Previously Stephan Richter wrote:
On Saturday 14 March 2009, Michael Howitz wrote:
Log message for revision 98113: set missing minimum version
Changed: U zope.app.component/trunk/setup.py
-=- Modified: zope.app.component/trunk/setup.py =================================================================== --- zope.app.component/trunk/setup.py 2009-03-14 19:56:47 UTC (rev 98112) +++ zope.app.component/trunk/setup.py 2009-03-14 20:14:47 UTC (rev 98113) @@ -78,7 +78,7 @@ 'zope.interface', 'zope.lifecycleevent', 'zope.location>3.4.0b1', - 'zope.publisher', + 'zope.publisher>=3.6.0', 'zope.schema', 'zope.security', 'zope.traversing',
Please, please, please no versions in setup.py.
If the package does not work with an older version of zope.publisher than imho that version restriction *has* to be in setup.py. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
On Sunday 15 March 2009, Wichert Akkerman wrote:
If the package does not work with an older version of zope.publisher than imho that version restriction *has* to be in setup.py.
And what if I backport the fix? We have done version specification like this in the Zope packages before and it almost brought development to complete halt, because versions would not match anymore. Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. "Zope Stephan Richter"
Hi Stephan, Wichert, Michael
Betreff: Re: [Zope3-checkins] [Checkins] SVN:zope.app.component/trunk/setup.py set missing minimum version
On Sunday 15 March 2009, Wichert Akkerman wrote:
If the package does not work with an older version of zope.publisher than imho that version restriction *has* to be in setup.py.
And what if I backport the fix?
We have done version specification like this in the Zope packages before and it almost brought development to complete halt, because versions would not match anymore.
+1 If one of several packages doesn't get updated with buildout is only possible if someone uses an index server which only have one of the both released packages available. If this is not true, then this is a bug in buildout and buildout doesn't catch the package versions correct. Which I think is not the case. All other reason I can think about that someone only got one package and another not if two package get released depends on a bad package management. Fix versions are a pain because you can't be sure that someone uses a patched or branched package which you will fix the version for. Package management should never be done in a single package! Michael, can you explain what's happen that you didn't got the zope.publisher release? And was this problem solved after fixing the version or did you do something else too? Regards Roger Ineichen
Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. "Zope Stephan Richter" _______________________________________________ Zope3-Checkins mailing list Zope3-Checkins@zope.org http://mail.zope.org/mailman/listinfo/zope3-checkins
Am 16.03.2009 um 03:53 schrieb Roger Ineichen:
Hi Stephan, Wichert, Michael
Betreff: Re: [Zope3-checkins] [Checkins] SVN:zope.app.component/trunk/setup.py set missing minimum version
On Sunday 15 March 2009, Wichert Akkerman wrote:
If the package does not work with an older version of zope.publisher than imho that version restriction *has* to be in setup.py.
And what if I back-port the fix?
We have done version specification like this in the Zope packages before and it almost brought development to complete halt, because versions would not match anymore.
+1
If one of several packages doesn't get updated with buildout is only possible if someone uses an index server which only have one of the both released packages available.
It is also possible if someone uses fixed versions in buildout (like me) and starts to upgrade versions of individual packages step by step to see if the current versions work together with the project code (as I did). It would be really helpful to know if an updated version requires new versions of other packages before seeing that the tests break. The version requirement produces is a much more explicit error message than an ImportError (which was the reason I added the minimum version). [...]
Michael, can you explain what's happen that you didn't got the zope.publisher release? And was this problem solved after fixing the version or did you do something else too?
I described above what I did. Fixing the version did not help me directly as this would require a new release of zope.app.component first, but it might help somebody else after a release was done. Yours sincerely, -- Michael Howitz · mh@gocept.com · software developer gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development
Previously Stephan Richter wrote:
On Sunday 15 March 2009, Wichert Akkerman wrote:
If the package does not work with an older version of zope.publisher than imho that version restriction *has* to be in setup.py.
And what if I backport the fix?
We have done version specification like this in the Zope packages before and it almost brought development to complete halt, because versions would not match anymore.
Than you adjust the dependency accordingly. I do not believe we should force the KGS on all users of zope packages, which is what you are effectively doing by never putting in version restrictions. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
Am 16.03.2009 um 03:33 schrieb Stephan Richter:
On Sunday 15 March 2009, Wichert Akkerman wrote:
If the package does not work with an older version of zope.publisher than imho that version restriction *has* to be in setup.py.
And what if I backport the fix?
In this case it was not a little fix which can be back-ported, it was a larger refactoring where a couple of packages where involved. Yours sincerely, -- Michael Howitz · mh@gocept.com · software developer gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development
Stephan Richter wrote:
On Sunday 15 March 2009, Wichert Akkerman wrote:
If the package does not work with an older version of zope.publisher than imho that version restriction *has* to be in setup.py.
And what if I backport the fix?
We have done version specification like this in the Zope packages before and it almost brought development to complete halt, because versions would not match anymore.
I'm not sure I agree you here, Stephan. A possible disagreement within the steering group, how interesting! :) I agree we should never hardcode version requirements in setup.py that limit the usable version to only one or a few selected ones. The version requirements in setup.py should always be "open". The most widely open requirement is this: zope.foo but another open requirement is this: zope.foo >= 1.3 I also don't recall open requirements bringing development to a halt? I think more specific open requirements (as opposed to the most widely open requirement) can be very useful. Useful to the maintainers of the Zope Framework KGS, the Zope 3 KGS, the Grok KGS, the Zope 2 KGS, and to application specific lists of packages, and users who are developing or testing packages in some other way. I've used this pattern quite successfully when developing a bunch of related packages. You bring up the case of backporting a fix (a relatively rare occurrence, but it certainly happens): So, zope.bar 3.2 says it needs zope.foo >= 1.3. Now you take something from zope.foo 1.3 and also put it in zope.foo 1.2.3 and onwards. zope.bar could now work with zope.foo 1.2 too, and it doesn't declare that it does. You could release a new minor version of zope.bar 3.2.1 that has a wider specification (if I get the spelling right): zope.foo >= 1.3, >= 1.2.3 (I'm not sure whether setuptools will consider this an "adjacent redundant condition and resolve this to zope.foo >= 1.2.3..) Updating that in all packages that depend on zope.foo for just that feature is indeed a bit of a burden. It does make it more likely that when someone does an update they'll get a set of package that work together. Opinions? Regards, Martijn
On Mon, Mar 16, 2009 at 10:24 AM, Martijn Faassen <faassen@startifact.com> wrote:
Stephan Richter wrote:
On Sunday 15 March 2009, Wichert Akkerman wrote:
If the package does not work with an older version of zope.publisher than imho that version restriction *has* to be in setup.py.
And what if I backport the fix?
We have done version specification like this in the Zope packages before and it almost brought development to complete halt, because versions would not match anymore.
I'm not sure I agree you here, Stephan. A possible disagreement within the steering group, how interesting! :)
I think more specific open requirements (as opposed to the most widely open requirement) can be very useful. Useful to the maintainers of the Zope Framework KGS, the Zope 3 KGS, the Grok KGS, the Zope 2 KGS, and to application specific lists of packages, and users who are developing or testing packages in some other way. I've used this pattern quite successfully when developing a bunch of related packages.
I don't like version requirements in setup.py because they assume too much about how people are using the package. Lets say that someone adds two bug fixes to zope.foo (call them fix A and fix B) and then does a release. Fix A requires zope.bar >= 1.5 and fix B doesn't. If I want to benefit from fix B in my app (and don't use the feature fix A repaired), then I shouldn't be forced to upgrade zope.bar. Another way to look at it: Just because a package's test suite won't pass without a particular version of a dependency doesn't mean that every user of the package needs that version of the dependency. If there were a way to ignore setup.py version requirements I'd be happy to have them and ignore them. Alternatively (my preference) the package would set versions in its buildout using the KGS against which it works (plus any exceptions). People could then refer to that to know what versions it actually works with and they can verify that it works by running the tests. -- Benji York Senior Software Engineer Zope Corporation
Am 16.03.2009 um 15:49 schrieb Benji York: [...]
I don't like version requirements in setup.py because they assume too much about how people are using the package.
Lets say that someone adds two bug fixes to zope.foo (call them fix A and fix B) and then does a release. Fix A requires zope.bar >= 1.5 and fix B doesn't. If I want to benefit from fix B in my app (and don't use the feature fix A repaired), then I shouldn't be forced to upgrade zope.bar.
+1 in principal, but this thread arose from adding a minimum version because the package failed with an ImportError in its main __init__.py Yours sincerely, -- Michael Howitz · mh@gocept.com · software developer gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development
Benji York wrote:
Lets say that someone adds two bug fixes to zope.foo (call them fix A and fix B) and then does a release. Fix A requires zope.bar >= 1.5 and fix B doesn't. If I want to benefit from fix B in my app (and don't use the feature fix A repaired), then I shouldn't be forced to upgrade zope.bar.
I don't think that's your call to make, it's the maintainer of zope.foo's call.
Another way to look at it: Just because a package's test suite won't pass without a particular version of a dependency doesn't mean that every user of the package needs that version of the dependency.
WRONG! If you use a package, you need to accept that its dependencies are decided by its author. Guessing that you know better than its author is a very dangerous game and leads to a path where you might as well not bother using a package manager at all and just go back to hit'n'hope...
If there were a way to ignore setup.py version requirements I'd be happy to have them and ignore them.
Wow...
Alternatively (my preference) the package would set versions in its buildout using the KGS against which it works (plus any exceptions).
KGS will be a zope dead-end imnsho... It's useful, but I don't see anyone outside the Zope community caring about it... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Previously Chris Withers wrote:
Benji York wrote:
Lets say that someone adds two bug fixes to zope.foo (call them fix A and fix B) and then does a release. Fix A requires zope.bar >= 1.5 and fix B doesn't. If I want to benefit from fix B in my app (and don't use the feature fix A repaired), then I shouldn't be forced to upgrade zope.bar.
I don't think that's your call to make, it's the maintainer of zope.foo's call.
Another way to look at it: Just because a package's test suite won't pass without a particular version of a dependency doesn't mean that every user of the package needs that version of the dependency.
WRONG! If you use a package, you need to accept that its dependencies are decided by its author. Guessing that you know better than its author is a very dangerous game and leads to a path where you might as well not bother using a package manager at all and just go back to hit'n'hope...
If there were a way to ignore setup.py version requirements I'd be happy to have them and ignore them.
Wow...
Alternatively (my preference) the package would set versions in its buildout using the KGS against which it works (plus any exceptions).
KGS will be a zope dead-end imnsho... It's useful, but I don't see anyone outside the Zope community caring about it...
This is an important point. As I see it the KGS is a Zope-only thing, and is just a workaround for the mindless behaviour of setuptools. I do not see it gaining acceptance outside of the Zope community, and imho effort is better spent on improving the packaging tools. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 23.03.2009 14:26 Uhr, Wichert Akkerman wrote:
This is an important point. As I see it the KGS is a Zope-only thing, and is just a workaround for the mindless behaviour of setuptools. I do not see it gaining acceptance outside of the Zope community, and imho effort is better spent on improving the packaging tools.
Are there other Python projects that have to deal with such a huge amount of packages and dependencies? I don't know any similar project. So the solution must come from the Zope world (which does not mean that we participate in the packaging toolchain development as Tarek does). Andreas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAknHjz0ACgkQCJIWIbr9KYwNUQCfY0vCy2sf/rmbCIUqFxiVq4a1 Hu4AmQFu63m10iPcU+j63eyzLV8WQJp9 =w4Ni -----END PGP SIGNATURE-----
Previously Andreas Jung wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 23.03.2009 14:26 Uhr, Wichert Akkerman wrote:
This is an important point. As I see it the KGS is a Zope-only thing, and is just a workaround for the mindless behaviour of setuptools. I do not see it gaining acceptance outside of the Zope community, and imho effort is better spent on improving the packaging tools.
Are there other Python projects that have to deal with such a huge amount of packages and dependencies? I don't know any similar project. So the solution must come from the Zope world (which does not mean that we participate in the packaging toolchain development as Tarek does).
I don't know. I just feel quite strongly that improving the packaging tools would be a better use of manpower than working on the KGS in its current form. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
Hi Wichert
Betreff: Re: [Zope-dev] setting missing minimum version in setup.py
Previously Andreas Jung wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 23.03.2009 14:26 Uhr, Wichert Akkerman wrote:
This is an important point. As I see it the KGS is a Zope-only thing, and is just a workaround for the mindless behaviour of setuptools. I do not see it gaining acceptance outside of
the Zope
community, and imho effort is better spent on improving the packaging tools.
Are there other Python projects that have to deal with such a huge amount of packages and dependencies? I don't know any similar project. So the solution must come from the Zope world (which does not mean that we participate in the packaging toolchain development as Tarek does).
I don't know. I just feel quite strongly that improving the packaging tools would be a better use of manpower than working on the KGS in its current form.
Stephan imlemented the KGS because there where a couple of limitations which we run into. I think the best you can do is to find out what they issues are which the KGS solves. After that you are very welcome to show us a better way to solve the same issues at a package lavel. Probably a way to go is to make both concept compatible with each other. Which probably means we should be able to ignore versions in packages if a KGS concept get used? (not sure if this is possible) Regards Roger Ineichen
Wichert.
-- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Roger Ineichen wrote:
Probably a way to go is to make both concept compatible with each other. Which probably means we should be able to ignore versions in packages if a KGS concept get used? (not sure if this is possible)
NO! This is INSANE! The version requirements in a package should be met by a KGS, it's just that they might be more rigid in a KGS... eg: KGS: foo.bar==1.7.1 In a package that depends on foo.bar: foo.bar >= 1.6 cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Wichert Akkerman wrote:
Are there other Python projects that have to deal with such a huge amount of packages and dependencies? I don't know any similar project. So the solution must come from the Zope world (which does not mean that we participate in the packaging toolchain development as Tarek does).
I don't know. I just feel quite strongly that improving the packaging tools would be a better use of manpower than working on the KGS in its current form.
Then dive in and help: - distutils-sig - catalog-sig - python-core :-) Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Hey, Wichert Akkerman wrote: [snip]
This is an important point. As I see it the KGS is a Zope-only thing, and is just a workaround for the mindless behaviour of setuptools. I do not see it gaining acceptance outside of the Zope community, and imho effort is better spent on improving the packaging tools.
KGS is two things: * KGS the software * KGS the concept KGS the concept will have a life outside of the Zope world. KGS the concept is very easy to implement; you just make available on some URL a buildout versions.cfg, or you run your own package index. I'll note that buildout seems to be clawing to a life outside of the Zope world; I know of several Django users that adopted it. It'd be much more popular if we had a good bunch of beginner level tutorials for it. Anyway, we've already decided to use setup.py dependencies for API compatibility. That's the best you're going to get out of us for a while. Regards, Martijn
Martijn Faassen wrote:
KGS is two things:
* KGS the software
* KGS the concept
KGS the concept will have a life outside of the Zope world.
I stand by my predication that even the KGS concept will never make it beyond Zope...
KGS the concept is very easy to implement; you just make available on some URL a buildout versions.cfg, or you run your own package index.
OK, the former I can see happening on an end-user project, the latter is just too much work.
I'll note that buildout seems to be clawing to a life outside of the Zope world; I know of several Django users that adopted it. It'd be much more popular if we had a good bunch of beginner level tutorials for it.
I promised to deliver this, and will do as soon as I get to that point in my work stack...
Anyway, we've already decided to use setup.py dependencies for API compatibility. That's the best you're going to get out of us for a while.
Good :-) Chris (although it's sad to note that Zope 2.12 can't be made to work without a KGS-like list :-/) -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote at 2009-4-2 20:42 +0100:
...
KGS the concept is very easy to implement; you just make available on some URL a buildout versions.cfg, or you run your own package index.
OK, the former I can see happening on an end-user project, the latter is just too much work.
Tres has earlier proposed a meta egg to represent "versions.cfg" in a setuptools only (non buildout) environment. A meta egg is an egg that only list dependencies and does not contain code of its own. -- Dieter
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dieter Maurer wrote:
Chris Withers wrote at 2009-4-2 20:42 +0100:
...
KGS the concept is very easy to implement; you just make available on some URL a buildout versions.cfg, or you run your own package index. OK, the former I can see happening on an end-user project, the latter is just too much work.
Not really. Collect the tarballs, run a script, configure Apache to serve that diretory. Probably less than half an hour per release, or less once the pattern is established, with the upside that builds against the "controlled" index are not susceptible to the breakage induced by varying levels of "release discipline" among the hundreds of folks whose pacakges are on PyPI.
Tres has earlier proposed a meta egg to represent "versions.cfg" in a setuptools only (non buildout) environment.
A meta egg is an egg that only list dependencies and does not contain code of its own.
I also intened that eggified-Zope2 releases would "pin" the underlying versions, so that installing it would give you a fixed configuration. Others overrode that choice, preferring to have an "upgradeable" version, which then requires a KGS. It might be sensible to distribute both variants, actually. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ2Myi+gerLs4ltQ4RAr6qAJ9f55E4qv/IPqqbf8ESW14s44e/DQCfaG/n E8o9/oqzetJnS/mz0fMfeKI= =8gpd -----END PGP SIGNATURE-----
Tres Seaver wrote:
KGS the concept is very easy to implement; you just make available on some URL a buildout versions.cfg, or you run your own package index. OK, the former I can see happening on an end-user project, the latter is just too much work.
Not really. Collect the tarballs, run a script, configure Apache to serve that diretory.
Hmm, too much... but is it needed? Can you not point index at just a local folder on disk? I'm sure the Plone folks did something like this, maybe Hanno can chime in?
A meta egg is an egg that only list dependencies and does not contain code of its own.
I also intened that eggified-Zope2 releases would "pin" the underlying versions, so that installing it would give you a fixed configuration.
I think this makes sense for whatever replaces the "tarball release".
Others overrode that choice, preferring to have an "upgradeable" version, which then requires a KGS. It might be sensible to distribute both variants, actually.
Yeah, I think a "naked egg" that pins the right versions and a "big fat tarball" which is absolutely nailed to only use the stuff it ships with makes sense... ...tricky to install "extra stuff" in the latter though. Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Previously Chris Withers wrote:
Tres Seaver wrote:
KGS the concept is very easy to implement; you just make available on some URL a buildout versions.cfg, or you run your own package index. OK, the former I can see happening on an end-user project, the latter is just too much work.
Not really. Collect the tarballs, run a script, configure Apache to serve that diretory.
Hmm, too much... but is it needed? Can you not point index at just a local folder on disk? I'm sure the Plone folks did something like this, maybe Hanno can chime in?
What we do is: collect the tarballs, serve the resulting directory. I have not seen a need to run a script. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Wichert Akkerman wrote:
Previously Chris Withers wrote:
Tres Seaver wrote:
KGS the concept is very easy to implement; you just make available on some URL a buildout versions.cfg, or you run your own package index. OK, the former I can see happening on an end-user project, the latter is just too much work. Not really. Collect the tarballs, run a script, configure Apache to serve that diretory. Hmm, too much... but is it needed? Can you not point index at just a local folder on disk? I'm sure the Plone folks did something like this, maybe Hanno can chime in?
What we do is: collect the tarballs, serve the resulting directory. I have not seen a need to run a script.
If you want to an index, rather than just a directory for 'find-links', you need to create the 'PyPI simple format' directory structure. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ2iu0+gerLs4ltQ4RAsbCAJ9T//C/+Ok1s4VgDQEXcnE+d6LHFQCfVTG8 +nE8OhQjjRncytfdRiDoIkw= =uXI+ -----END PGP SIGNATURE-----
Previously Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Wichert Akkerman wrote:
Previously Chris Withers wrote:
Tres Seaver wrote:
> KGS the > concept is very easy to implement; you just make available on some URL a > buildout versions.cfg, or you run your own package index. OK, the former I can see happening on an end-user project, the latter is just too much work. Not really. Collect the tarballs, run a script, configure Apache to serve that diretory. Hmm, too much... but is it needed? Can you not point index at just a local folder on disk? I'm sure the Plone folks did something like this, maybe Hanno can chime in?
What we do is: collect the tarballs, serve the resulting directory. I have not seen a need to run a script.
If you want to an index, rather than just a directory for 'find-links', you need to create the 'PyPI simple format' directory structure.
What is the advantage of creating an index? Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Wichert Akkerman wrote:
Previously Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Wichert Akkerman wrote:
Previously Chris Withers wrote:
Tres Seaver wrote:
>> KGS the >> concept is very easy to implement; you just make available on some URL a >> buildout versions.cfg, or you run your own package index. > OK, the former I can see happening on an end-user project, the latter is > just too much work. Not really. Collect the tarballs, run a script, configure Apache to serve that diretory. Hmm, too much... but is it needed? Can you not point index at just a local folder on disk? I'm sure the Plone folks did something like this, maybe Hanno can chime in? What we do is: collect the tarballs, serve the resulting directory. I have not seen a need to run a script. If you want to an index, rather than just a directory for 'find-links', you need to create the 'PyPI simple format' directory structure.
What is the advantage of creating an index?
Not relying on PyPI. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ2krs+gerLs4ltQ4RAty5AKDYSPuvEieNqBm1TnyiNk7ulJzYHwCgugTv xoKaPWWpvkrVsQH0IqPokCQ= =YGGd -----END PGP SIGNATURE-----
Wichert Akkerman wrote:
Previously Chris Withers wrote:
Tres Seaver wrote:
KGS the concept is very easy to implement; you just make available on some URL a buildout versions.cfg, or you run your own package index. OK, the former I can see happening on an end-user project, the latter is just too much work. Not really. Collect the tarballs, run a script, configure Apache to serve that diretory. Hmm, too much... but is it needed? Can you not point index at just a local folder on disk? I'm sure the Plone folks did something like this, maybe Hanno can chime in?
What we do is: collect the tarballs, serve the resulting directory. I have not seen a need to run a script.
How do you collect the tarballs? How do you serve the resulting directory? Chris
Previously Chris Withers wrote:
Wichert Akkerman wrote:
What we do is: collect the tarballs, serve the resulting directory. I have not seen a need to run a script.
How do you collect the tarballs?
buildout download cache
How do you serve the resulting directory?
standard apache directory listing: http://dist.plone.org/release/<version number> Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote:
Wichert Akkerman wrote:
Previously Chris Withers wrote:
Tres Seaver wrote:
> KGS the > concept is very easy to implement; you just make available on some URL a > buildout versions.cfg, or you run your own package index. OK, the former I can see happening on an end-user project, the latter is just too much work. Not really. Collect the tarballs, run a script, configure Apache to serve that diretory. Hmm, too much... but is it needed? Can you not point index at just a local folder on disk? I'm sure the Plone folks did something like this, maybe Hanno can chime in? What we do is: collect the tarballs, serve the resulting directory. I have not seen a need to run a script.
How do you collect the tarballs?
I dig them out of wherever I told whatver tool (buildout, compoze fetch, etc) to put them.
How do you serve the resulting directory?
See above. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ3JKI+gerLs4ltQ4RAtZ8AJoCjuoGAdIeHBd+A/16jms+U+ItGgCdEA6d MAm43IFFskpvj8sANIPGvwA= =P5rk -----END PGP SIGNATURE-----
Dieter Maurer wrote:
Tres has earlier proposed a meta egg to represent "versions.cfg" in a setuptools only (non buildout) environment.
A meta egg is an egg that only list dependencies and does not contain code of its own.
Indeed, so we'd need 2 eggs for Zope 2 :-( Something I bumped into recently: If the Zope 2 egg actually hard-specifies its dependencies, how do I upgrade any of those dependencies to newer versions that fix bugs i may have encountered/fixed? With buildout-only, it's easy, just override the versions in the versions section of the topmost buildout.cfg. How would this be done if the Zope 2 egg or meta-egg hard-specifies the versions? Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Sat, Apr 11, 2009 at 13:12, Chris Withers <chris@simplistix.co.uk> wrote:
How would this be done if the Zope 2 egg or meta-egg hard-specifies the versions?
You can just install that egg that needs updating, no? I'm not sure... in worst case, you need to wait for update, just like you did before it was eggyfied. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart Regebro wrote:
On Sat, Apr 11, 2009 at 13:12, Chris Withers <chris@simplistix.co.uk> wrote:
How would this be done if the Zope 2 egg or meta-egg hard-specifies the versions?
You can just install that egg that needs updating, no?
How, if I'm using buildout? How, if I'm using easy_install or pip?
in worst case, you need to wait for update, just like you did before it was eggyfied.
No, before I could always stick a new version manually in lib/python of my instance. Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Sat, Apr 11, 2009 at 15:48, Chris Withers <chris@simplistix.co.uk> wrote:
You can just install that egg that needs updating, no?
How, if I'm using buildout?
I quote yourself: "With buildout-only, it's easy, just override the versions in the versions section of the topmost buildout.cfg."
How, if I'm using easy_install or pip?
Does easy_install keep track of already installed dependencies and refuse to install it if it break dependencies? If yes, then i don't know, if no, then you can just install the version you want.
in worst case, you need to wait for update, just like you did before it was eggyfied.
No, before I could always stick a new version manually in lib/python of my instance.
Why would you not be able to do that now? I evidently don't understand the problem. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart Regebro wrote at 2009-4-11 16:12 +0200:
... Does easy_install keep track of already installed dependencies and refuse to install it if it break dependencies?
"easy_install" checks dependencies only at installation time -- for the egg that is installed (not for those that are already installed) .
If yes, then i don't know, if no, then you can just install the version you want.
We are in the "no" case. -- Dieter
On Sun, Apr 12, 2009 at 08:56, Dieter Maurer <dieter@handshake.de> wrote:
Lennart Regebro wrote at 2009-4-11 16:12 +0200:
... Does easy_install keep track of already installed dependencies and refuse to install it if it break dependencies?
"easy_install" checks dependencies only at installation time -- for the egg that is installed (not for those that are already installed) .
If yes, then i don't know, if no, then you can just install the version you want.
We are in the "no" case.
No, you just said above we were in the "yes" case. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Sun, Apr 12, 2009 at 17:26, Lennart Regebro <regebro@gmail.com> wrote:
No, you just said above we were in the "yes" case.
No you didn't. My bad. We are in the no case. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Chris Withers wrote:
Dieter Maurer wrote:
Tres has earlier proposed a meta egg to represent "versions.cfg" in a setuptools only (non buildout) environment.
A meta egg is an egg that only list dependencies and does not contain code of its own.
Indeed, so we'd need 2 eggs for Zope 2 :-(
Something I bumped into recently:
If the Zope 2 egg actually hard-specifies its dependencies, how do I upgrade any of those dependencies to newer versions that fix bugs i may have encountered/fixed?
You roll your own meta egg. You cannot override hard version specifications inside a setup.py. In all other debates we seemed to agree on not over specifying requirements in setup.py files, I wonder why anybody still tries to follow this route. Let's see what other people who want to support easy_install do: They require you to use virtualenv and create a links page, which has all the dependencies in all the required versions on it: http://pylonshq.com/download/0.9.7/ http://www.turbogears.org/2.0/downloads/2.0final/ http://dist.repoze.org/zope2/dev/ http://dist.plone.org/release/3.3rc2/ In order to install from on of these you do for example: easy_install -f http://pylonshq.com/download/0.9.7 Pylons and that's it. Some projects package up the multiple steps (including installing setuptools) into a little helper script. As you might notice all that is required is a known URL and Apache serving a bunch of files directly. Hanno
Hanno Schlichting wrote:
In all other debates we seemed to agree on not over specifying requirements in setup.py files, I wonder why anybody still tries to follow this route.
Tres argues for the easy_install case. If you don't have a meta-egg that does the same as a versions section in a buildout, how do you get the same versions each time? If you *do* have a meta-egg, how do you override just one egg that you need to replace for, say, a security fix?
Let's see what other people who want to support easy_install do: They require you to use virtualenv and create a links page, which has all the dependencies in all the required versions on it:
http://pylonshq.com/download/0.9.7/ http://www.turbogears.org/2.0/downloads/2.0final/ http://dist.repoze.org/zope2/dev/ http://dist.plone.org/release/3.3rc2/
In order to install from on of these you do for example:
easy_install -f http://pylonshq.com/download/0.9.7 Pylons
and that's it. Some projects package up the multiple steps (including installing setuptools) into a little helper script.
As you might notice all that is required is a known URL and Apache serving a bunch of files directly.
Fair enough... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hanno Schlichting wrote:
Chris Withers wrote:
Dieter Maurer wrote:
Tres has earlier proposed a meta egg to represent "versions.cfg" in a setuptools only (non buildout) environment.
A meta egg is an egg that only list dependencies and does not contain code of its own. Indeed, so we'd need 2 eggs for Zope 2 :-(
Something I bumped into recently:
If the Zope 2 egg actually hard-specifies its dependencies, how do I upgrade any of those dependencies to newer versions that fix bugs i may have encountered/fixed?
You roll your own meta egg. You cannot override hard version specifications inside a setup.py.
In all other debates we seemed to agree on not over specifying requirements in setup.py files, I wonder why anybody still tries to follow this route.
Maybe because it makes for a predictable / reproducible installation? After all, the 2.12a1 egg already released is now broken under easy_isntall becasse it *doesn't* nail down its dependencies. Predicatbility requires one of the following: - - Run an index, and make people install from there rather than PyPI; at that point, you are a "distribution" maintainer, and can ensure consistency. Users who trust you can "safely" upgrade at will, because you are responsible for ensuring that the eggs in the index work together. - - Release a nailed-down egg (or the equivalent buildout.cfg / versions.cfg, and let people install from PyPI. Upgrades are not really safe: you won't get borked by newly released eggs, but removed / re-uploaded eggs can still hose you. - - Release some kind of monolith, generated from the same metadata as above. Upgrades aren't really possible here: you have to release a new version of the monolith.
Let's see what other people who want to support easy_install do: They require you to use virtualenv and create a links page, which has all the dependencies in all the required versions on it:
http://pylonshq.com/download/0.9.7/ http://www.turbogears.org/2.0/downloads/2.0final/ http://dist.repoze.org/zope2/dev/ http://dist.plone.org/release/3.3rc2/
In order to install from on of these you do for example:
easy_install -f http://pylonshq.com/download/0.9.7 Pylons
You want '-i' and the index URL, not '-f' and the "directory-containing- tarballs" URL.
and that's it. Some projects package up the multiple steps (including installing setuptools) into a little helper script.
As you might notice all that is required is a known URL and Apache serving a bunch of files directly.
Without supplying the '-i' ('--index-url') flag, you are still subject to the variability of PyPI. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ4NRD+gerLs4ltQ4RAjepAKCRsCzEHTcOnPpqE53Cs8Q0+WSdxACfZ7Zq LupGqM494WgV5QMAWl4X/HY= =NCYV -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11.04.2009 19:32 Uhr, Tres Seaver wrote:
Hanno Schlichting wrote:
Chris Withers wrote:
Dieter Maurer wrote:
Tres has earlier proposed a meta egg to represent "versions.cfg" in a setuptools only (non buildout) environment.
A meta egg is an egg that only list dependencies and does not contain code of its own. Indeed, so we'd need 2 eggs for Zope 2 :-(
Something I bumped into recently:
If the Zope 2 egg actually hard-specifies its dependencies, how do I upgrade any of those dependencies to newer versions that fix bugs i may have encountered/fixed? You roll your own meta egg. You cannot override hard version specifications inside a setup.py.
In all other debates we seemed to agree on not over specifying requirements in setup.py files, I wonder why anybody still tries to follow this route.
Maybe because it makes for a predictable / reproducible installation? After all, the 2.12a1 egg already released is now broken under easy_isntall becasse it *doesn't* nail down its dependencies.
Predicatbility requires one of the following:
- Run an index, and make people install from there rather than PyPI; at that point, you are a "distribution" maintainer, and can ensure consistency. Users who trust you can "safely" upgrade at will, because you are responsible for ensuring that the eggs in the index work together.
- Release a nailed-down egg (or the equivalent buildout.cfg / versions.cfg, and let people install from PyPI. Upgrades are not really safe: you won't get borked by newly released eggs, but removed / re-uploaded eggs can still hose you.
- Release some kind of monolith, generated from the same metadata as above. Upgrades aren't really possible here: you have to release a new version of the monolith.
Let's see what other people who want to support easy_install do: They require you to use virtualenv and create a links page, which has all the dependencies in all the required versions on it:
http://pylonshq.com/download/0.9.7/ http://www.turbogears.org/2.0/downloads/2.0final/ http://dist.repoze.org/zope2/dev/ http://dist.plone.org/release/3.3rc2/
In order to install from on of these you do for example:
easy_install -f http://pylonshq.com/download/0.9.7 Pylons
You want '-i' and the index URL, not '-f' and the "directory-containing- tarballs" URL.
and that's it. Some projects package up the multiple steps (including installing setuptools) into a little helper script.
As you might notice all that is required is a known URL and Apache serving a bunch of files directly.
Without supplying the '-i' ('--index-url') flag, you are still subject to the variability of PyPI.
A dedicated index is a clean solution (I have not figure out or understand how this is accomplished for Zope 3). Another low hanging fruit is releasing Zope 2 as 2 packages (one with unpinned versions (for buildout) and one with fully pinned versions). I am trying out this approach for the next a2 release. Andreas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkniFb8ACgkQCJIWIbr9KYwKWACdEa+VpOliCYnd8M1BLvVFJyHe 70wAoM8XlOrP7PBY0Bxqpf5Lev4GHVSh =FBm9 -----END PGP SIGNATURE-----
Hanno Schlichting wrote at 2009-4-11 14:35 +0200:
... In all other debates we seemed to agree on not over specifying requirements in setup.py files, I wonder why anybody still tries to follow this route.
Because the way the Zope 2.12a1 egg did it has broken within a few weeks.... -- Dieter
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote:
Dieter Maurer wrote:
Tres has earlier proposed a meta egg to represent "versions.cfg" in a setuptools only (non buildout) environment.
A meta egg is an egg that only list dependencies and does not contain code of its own.
Indeed, so we'd need 2 eggs for Zope 2 :-(
Something I bumped into recently:
If the Zope 2 egg actually hard-specifies its dependencies, how do I upgrade any of those dependencies to newer versions that fix bugs i may have encountered/fixed?
You roll another version of the meta-egg with updated dependencies (the meta-egg takes the place of the KGS). This isn't any harder than updating the buildout.cfg / versions.cfg, assuming you maintain the meta-egg which corresponds to your needs in SVN someplace. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ4NJI+gerLs4ltQ4RAmYWAKCW2rH0QnP5eLB3gEigZ0aFN53y8ACgkV1q vaUreNgVdJj1kdjCwQQxv/M= =DAcs -----END PGP SIGNATURE-----
On Monday 16 March 2009, Martijn Faassen wrote:
I'm not sure I agree you here, Stephan. A possible disagreement within the steering group, how interesting! :)
:-)
The most widely open requirement is this:
zope.foo
but another open requirement is this:
zope.foo >= 1.3
Sure, but here is what happened before. Person A made a bugfix to zope.foo 1.3, releasing zope.foo 1.3.1. He then said, ok, zope.bar 1.0.0 depends on zope.foo >= 1.3.1. Person B backports the bug fix to zope.foo 1.2.1. Now zope.bar would also work with 1.2.1, but can't because of the version specification. So person B has the choice of upgrading to zope.foo 1.3.x or release a new version of zope.bar. Updgrading to zope.foo 1.3.x might not be easy for various reasons that I think most of us experienced (I know I did). Releasing a new zope.bar version might not be possible, if person B does not have access. Also expecting person B to create new releases for all packages where the bug fix would work is unrealistic.
I also don't recall open requirements bringing development to a halt?
They did. :-)
You bring up the case of backporting a fix (a relatively rare occurrence, but it certainly happens):
Not so rare at all, I think.
Updating that in all packages that depend on zope.foo for just that feature is indeed a bit of a burden. It does make it more likely that when someone does an update they'll get a set of package that work together.
I think we have seen this is a no-go. There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-) Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. "Zope Stephan Richter"
Hi
Betreff: Re: [Zope-dev] setting missing minimum version in setup.py
On Monday 16 March 2009, Martijn Faassen wrote:
I'm not sure I agree you here, Stephan. A possible disagreement within the steering group, how interesting! :)
:-)
The most widely open requirement is this:
zope.foo
but another open requirement is this:
zope.foo >= 1.3
Sure, but here is what happened before.
Person A made a bugfix to zope.foo 1.3, releasing zope.foo 1.3.1. He then said, ok, zope.bar 1.0.0 depends on zope.foo
= 1.3.1. Person B backports the bug fix to zope.foo 1.2.1. Now zope.bar would also work with 1.2.1, but can't because of the version specification. So person B has the choice of upgrading to zope.foo 1.3.x or release a new version of zope.bar.
Updgrading to zope.foo 1.3.x might not be easy for various reasons that I think most of us experienced (I know I did). Releasing a new zope.bar version might not be possible, if person B does not have access. Also expecting person B to create new releases for all packages where the bug fix would work is unrealistic.
I also don't recall open requirements bringing development to a halt?
They did. :-)
You bring up the case of backporting a fix (a relatively rare occurrence, but it certainly happens):
Not so rare at all, I think.
Even if it's rare, why should we not support that? The consequence of fixing versions is to skip backporting. There is no way to have both. Are you really sure we like to skip backporting. Regards Roger Ineichen
Hey, Roger Ineichen wrote: [snip]
Even if it's rare, why should we not support that?
The consequence of fixing versions is to skip backporting. There is no way to have both. Are you really sure we like to skip backporting.
I haven't a clear idea about how often we backport and even less an idea on how often we'd *want* to backport. :) If, as Stephan was suggesting as a possible compromise, we try this for feature changes only, we'd only run into this issue if we start backporting features. Regards, Martijn
Roger Ineichen wrote:
The consequence of fixing versions is to skip backporting. There is no way to have both.
Rubbish. Martijn already showed what would need to happen here: the package specifying the depenedency needs a quick, 3rd point release to add the backported releases as suitable. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Hey, Stephan Richter wrote: [snip]
There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-)
Yes, I was thinking in this direction too. I can see this as more of an issue with bug fixes than with feature changes. This means that requirements can only say zope.foo >= x.y, and never zope.foo >= x.y.z. What do people think? Regards, Martijn
2009/3/16 Martijn Faassen <faassen@startifact.com>:
There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-)
Yes, I was thinking in this direction too. I can see this as more of an issue with bug fixes than with feature changes. This means that requirements can only say zope.foo >= x.y, and never zope.foo >= x.y.z.
What do people think?
That looks sane, so I'm +1 :) -- WBR, Dan Korostelev
On Mar 16, 2009, at 12:05 PM, Dan Korostelev wrote:
2009/3/16 Martijn Faassen <faassen@startifact.com>:
There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-)
Yes, I was thinking in this direction too. I can see this as more of an issue with bug fixes than with feature changes. This means that requirements can only say zope.foo >= x.y, and never zope.foo >= x.y.z.
What do people think?
That looks sane, so I'm +1 :)
Also +1 Gary
Am 16.03.2009 um 16:56 schrieb Martijn Faassen:
Hey,
Stephan Richter wrote: [snip]
There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-)
Yes, I was thinking in this direction too. I can see this as more of an issue with bug fixes than with feature changes. This means that requirements can only say zope.foo >= x.y, and never zope.foo >= x.y.z.
What do people think?
Sounds reasonable: +1 Yours sincerely, -- Michael Howitz · mh@gocept.com · software developer gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development
Previously Martijn Faassen wrote:
Hey,
Stephan Richter wrote: [snip]
There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-)
Yes, I was thinking in this direction too. I can see this as more of an issue with bug fixes than with feature changes. This means that requirements can only say zope.foo >= x.y, and never zope.foo >= x.y.z.
What do people think?
-1 still If I install a package I want to have a guarantee all aspects of that package work correctly. With this compromise that is not possible. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
Wichert Akkerman wrote:
Previously Martijn Faassen wrote:
Hey,
Stephan Richter wrote: [snip]
There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-)
Yes, I was thinking in this direction too. I can see this as more of an issue with bug fixes than with feature changes. This means that requirements can only say zope.foo >= x.y, and never zope.foo >= x.y.z.
What do people think?
-1 still
If I install a package I want to have a guarantee all aspects of that package work correctly. With this compromise that is not possible.
Wichert.
I am not quite sure what you mean... Are you saying that the proposal does not go far enough, and we should allow the full >=x.y.z? Or are you against any and all versions in setup.py? I think the best policy is to use versions specs for base packages (minimum, as well as maximum), but only when it is *known* that not meeting the requirements means the derived package is useless. This is most likely to happen when the API of the base package is changed (which is only allowed in a major version), but could in rare cases happen for other reasons. To me the important thing is not the major/minor version distinction, but rather the degree of uselessness. Of course there may be broad guidelines such as "only use major versions", but the way I see it this really calls for a decision on a case-by-case basis. Jacob
Previously Jacob Holm wrote:
Wichert Akkerman wrote:
Previously Martijn Faassen wrote:
Hey,
Stephan Richter wrote: [snip]
There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-)
Yes, I was thinking in this direction too. I can see this as more of an issue with bug fixes than with feature changes. This means that requirements can only say zope.foo >= x.y, and never zope.foo >= x.y.z.
What do people think?
-1 still
If I install a package I want to have a guarantee all aspects of that package work correctly. With this compromise that is not possible.
Wichert.
I am not quite sure what you mean... Are you saying that the proposal does not go far enough, and we should allow the full >=x.y.z? Or are you against any and all versions in setup.py?
I see no useful different between x.y and x.y.z here. All I want is if someone installs one of our packages that package will work as expected. If a package will only work with a certain revisions of a dependent package it has to state say. If we do not do that we are making it very hard for people to use Zope packages: we are effectively telling them that we make no guarantee about package stability and they will have to either use one of our many KGS or figure out all dependencies by hand. I do not think that is an acceptable message, and it certainly will not encourage people to use Zope packages. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
Wichert Akkerman wrote: [snip]
I see no useful different between x.y and x.y.z here. All I want is if someone installs one of our packages that package will work as expected. If a package will only work with a certain revisions of a dependent package it has to state say.
I do see a useful difference between the two. x.y is a feature release that might have changed the API or behavior. If you rely on this in a version of your package, you will have to indicate that this is so. x.y.z is a bugfix release. If we do it right, there will be no change in the API and only small changes in misbehavior. Therefore it seems far less likely to me that a package ends *needing* to depend on a minimum version. In addition, porting back bugfix releases is far harder. We should have a good communication channels so that people maintaining version lists can be made aware of bugfixes though. Anyway, I'm closing this discussion for now as we are going to try it for feature releases for a while, see how that goes. I've just recorded it in the decisions document. It should be a lot better for you than us doing nothing, and I'm afraid in the old zope-dev it's what you would've gotten. So rejoice! :) Regards, Martijn
Previously Martijn Faassen wrote:
Wichert Akkerman wrote: [snip]
I see no useful different between x.y and x.y.z here. All I want is if someone installs one of our packages that package will work as expected. If a package will only work with a certain revisions of a dependent package it has to state say.
I do see a useful difference between the two.
x.y is a feature release that might have changed the API or behavior. If you rely on this in a version of your package, you will have to indicate that this is so.
x.y.z is a bugfix release. If we do it right, there will be no change in the API and only small changes in misbehavior. Therefore it seems far less likely to me that a package ends *needing* to depend on a minimum version. In addition, porting back bugfix releases is far harder.
If version x.y of package A adds a new feature which requires a feature in package B, but was broken in B until version d.e.f of that package, I would expect version x.y of package A to have a dependency on version d.e.f of package B. Do you agree with that? Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
Hi Wichert, Steering Group?
Betreff: Re: [Zope-dev] setting missing minimum version in setup.py
Previously Martijn Faassen wrote:
Wichert Akkerman wrote: [snip]
I see no useful different between x.y and x.y.z here. All I want is if someone installs one of our packages that package will work as expected. If a package will only work with a certain revisions of a dependent package it has to state say.
I do see a useful difference between the two.
x.y is a feature release that might have changed the API or behavior. If you rely on this in a version of your package, you will have to indicate that this is so.
x.y.z is a bugfix release. If we do it right, there will be no change in the API and only small changes in misbehavior. Therefore it seems far less likely to me that a package ends *needing* to depend on a minimum version. In addition, porting back bugfix releases is far harder.
If version x.y of package A adds a new feature which requires a feature in package B, but was broken in B until version d.e.f of that package, I would expect version x.y of package A to have a dependency on version d.e.f of package B. Do you agree with that?
What do you do if version x.y works with d.e.d but not with d.e.e (because it's borken) and fixed in d.e.f. This means you could use d.e.d or d.e.f. but not d.e.e What's your solution then? Fix the version to d.e.d or d.e.f or skip fixing versions? This is a use case where fixing versions in packages doesn't work and the KGS only can help. You can have different KGS version indexes. One with d.e.d and one with d.e.f. This is the benefit of the KGS. You can define whatever you like an nobody is able to block packages which you need to use. Note; Not that fixing versions in a package is a bad thing in general. But such fixed versions will mess up the KGS that's the bad part. And since the KGS solves any kind of version conflict problems, there is no way to do it in a package anymore. I think you are suggesting that we should be able to use all package without a KGS. If so,you are right, then we need fixed versions. But if we decide to use KGS as our primary pattern, then we do not need to mess up with versions in packages. Steering group? Do we support a release management at package level or do we only guarantee a working release/version management based on KGS? Regards Roger Ineichen
Wichert.
-- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Roger Ineichen wrote:
What do you do if version x.y works with d.e.d but not with d.e.e (because it's borken) and fixed in d.e.f.
This means you could use d.e.d or d.e.f. but not d.e.e
What's your solution then? Fix the version to d.e.d or d.e.f or skip fixing versions?
The version requirements in setup.py should specify only API compatibility. They have nothing to do with bug fixes; that's the domain of the KGS. How about an example. Let's say we have zope.schema version 5.2, which requires zope.interface version 4.1 because zope.interface version 4.0 has a different API. Let's also say that zope.interface version 4.1.5 has a bug that breaks zope.schema, but versions 4.1.4 and 4.1.6 are OK. Since that level of detail is burdensome and unnecessary for setup.py files, zope.schema should require "zope.interface >= 4.1" and let the KGS provide the finer detail. People who don't use a KGS will be fine because they'll get 4.1.6+ as soon as it's released.
Note; Not that fixing versions in a package is a bad thing in general. But such fixed versions will mess up the KGS that's the bad part. And since the KGS solves any kind of version conflict problems, there is no way to do it in a package anymore.
The KGS is too fine grained for many uses. I, for example, would like to use the latest z3c.form, but I really can't, because it relies on a bunch of other packages with revised APIs and the API requirements are not written anywhere. I don't care about bugs; I can fix bugs. What I can't do is read other developer's minds to discover API version requirements. Note that if I could use the current z3c.form, I would also be able to help develop it. Surely many other developers face similar circumstances. Thus I believe the Zope project is quietly missing opportunities for developers to get involved due to the lack of coarse-grained version requirements. Shane
On Tuesday 17 March 2009, Shane Hathaway wrote:
The version requirements in setup.py should specify only API compatibility. They have nothing to do with bug fixes; that's the domain of the KGS. How about an example.
Yes, that's a good summary of what we agreed on. The more I think about it, the more I am convinced this is the correct way. Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. "Zope Stephan Richter"
Stephan Richter wrote:
On Tuesday 17 March 2009, Shane Hathaway wrote:
The version requirements in setup.py should specify only API compatibility. They have nothing to do with bug fixes; that's the domain of the KGS. How about an example.
Yes, that's a good summary of what we agreed on. The more I think about it, the more I am convinced this is the correct way.
Agreed. I'll record this clarification, thanks Shane. Regards, Martijn
Roger Ineichen wrote:
What do you do if version x.y works with d.e.d but not with d.e.e (because it's borken) and fixed in d.e.f.
You release x.y.1 which has dependencies on d.e.d, >=d.e.f.
This is a use case where fixing versions in packages doesn't work
Sure it does.
This is the benefit of the KGS. You can define whatever you like an nobody is able to block packages which you need to use.
Oh? How so? Surely a KGS locks everything down?! cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Martijn Faassen wrote:
x.y.z is a bugfix release. If we do it right, there will be no change in the API and only small changes in misbehavior. Therefore it seems far less likely to me that a package ends *needing* to depend on a minimum version.
I don't agree. If your package hsa bugs reported by its users, and the bug is caused by a bug in a dependent package, the bugfix release of your package might well be just to add a test and then change a requirement from, saya, >=1.3 to >=1.3.2.
Anyway, I'm closing this discussion for now as we are going to try it for feature releases for a while, see how that goes.
Oh, sorry, reading and replying from top to bottom of this thread so only just seen this. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Wichert Akkerman wrote:
I see no useful different between x.y and x.y.z here. All I want is if someone installs one of our packages that package will work as expected. If a package will only work with a certain revisions of a dependent package it has to state say.
Yes.
If we do not do that we are making it very hard for people to use Zope packages: we are effectively telling them that we make no guarantee about package stability and they will have to either use one of our many KGS or figure out all dependencies by hand. I do not think that is an acceptable message, and it certainly will not encourage people to use Zope packages.
Exactly. Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Stephan Richter wrote:
Updgrading to zope.foo 1.3.x might not be easy for various reasons that I think most of us experienced (I know I did). Releasing a new zope.bar version might not be possible, if person B does not have access.
If a fix is possible, and someone backports it, a release should be made. If person B does not have access, there's nothing stopping them cutting their own egg and putting it up on their own egg server. I know I've had to do this before for packages like xlrd, which often have development but are extremely rarely released, and have maintainers who don't care about eggs at all.
Also expecting person B to create new releases for all packages where the bug fix would work is unrealistic.
Indeed, they only need to create releases for things they care about, it's the open source way :-)
I also don't recall open requirements bringing development to a halt?
They did. :-)
I'd love to see some actualy evidence of that ;-)
Updating that in all packages that depend on zope.foo for just that feature is indeed a bit of a burden. It does make it more likely that when someone does an update they'll get a set of package that work together.
I think we have seen this is a no-go.
Oh?
There is a compromise I am willing to take. If package zope.bar depends on a *new feature* or *feature change* in zope.foo 1.3.x, then it should specify the version. In other words specifying open restrictions on the major version levels is okay, but never on the bug fix level. (I just hope this does not bite us later. ;-)
I'm -1 on this compromise. If a package needs a bugfix of another package to work, then it should specify it. However, I *am* -1 on specifying a later version of a package just because it's available ;-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Martijn Faassen wrote:
The version requirements in setup.py should always be "open".
The most widely open requirement is this:
zope.foo
but another open requirement is this:
zope.foo >= 1.3
I also don't recall open requirements bringing development to a halt?
I think more specific open requirements (as opposed to the most widely open requirement) can be very useful. Useful to the maintainers of the Zope Framework KGS, the Zope 3 KGS, the Grok KGS, the Zope 2 KGS, and to application specific lists of packages, and users who are developing or testing packages in some other way. I've used this pattern quite successfully when developing a bunch of related packages.
You bring up the case of backporting a fix (a relatively rare occurrence, but it certainly happens):
So, zope.bar 3.2 says it needs zope.foo >= 1.3.
Now you take something from zope.foo 1.3 and also put it in zope.foo 1.2.3 and onwards. zope.bar could now work with zope.foo 1.2 too, and it doesn't declare that it does.
You could release a new minor version of zope.bar 3.2.1 that has a wider specification (if I get the spelling right):
zope.foo >= 1.3, >= 1.2.3
(I'm not sure whether setuptools will consider this an "adjacent redundant condition and resolve this to zope.foo >= 1.2.3..)
Updating that in all packages that depend on zope.foo for just that feature is indeed a bit of a burden. It does make it more likely that when someone does an update they'll get a set of package that work together.
Opinions?
However you need to spell it, what you suggest is correct. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (16)
-
Andreas Jung -
Benji York -
Chris Withers -
Dan Korostelev -
Dieter Maurer -
Gary Poster -
Hanno Schlichting -
Jacob Holm -
Lennart Regebro -
Martijn Faassen -
Michael Howitz -
Roger Ineichen -
Shane Hathaway -
Stephan Richter -
Tres Seaver -
Wichert Akkerman