On Dec 8, 2008, at 10:52 AM, Benji York wrote:

On Mon, Dec 8, 2008 at 9:18 AM, Gary Poster <gary.poster@gmail.com> wrote:

...

On Dec 8, 2008, at 9:02 AM, Benji York wrote:

...

On Sat, Dec 6, 2008 at 10:28 AM, Christian Zagrodnick <cz@gocept.com> wrote:

...

Log message for revision 93722: - Switched to Zope 3.4 KGS.

- New lines are no longer stripped in XML and HTML code contained in a textarea; requires ClientForm >= 0.2.10 (LP #268139).

This revision make the buildout fail with

Error: Couldn't find a distribution for 'ClientForm>=0.2.10'.

I suspect you had that version of ClientForm in your cache and didn't realize that it is not available in the KGS index.

Even if we fixed that, I don't want to require a particular version of ClientForm in testbrowser. There's no need to impose a newer version on people who don't need it. Anyone who does need the bug fix can specify the newer version in their project.

FWIW, I disagree. The specification that you removed is exactly the sort of thing that I think is appropriate in setup.py. The tests will now fail (I assume, since I believe Christian Z added testbrowser tests for the failure caused by the ClientForm bug) with a lower version of ClientForm, so it is appropriate to set the value in setup.py.

Nope, the tests will pass in the zope.testbrowser buildout.

That's not what I said. :-)

However, if a testbrowser user for some reason run the testbrowser tests outside of its buildout, then you're right, they may see a failure if their versions aren't new enough. At that point I would hope they would read the CHANGES.txt and see that a new version is required.

The trade off is in requiring people to upgrade a dependency in order to get a bug fix that they may not care about.

In the large, this is the problem with specifying versions in setup.py. Doing so assumes too much about how people are using all the dependencies involved.

Here's a scenario: I fix a bug in some package, it depends on a newer version of say, zope.publisher. I put that requirement in my package's setup.py. A user of my package upgrades to get an unrelated bug fix and is forced to use the newer zope.publisher. It so happens that that the new version of zope.publisher has a different bug that bites them. They now are in a bad spot.

If the setup.py hadn't specified a version then the programmer could have constructed a set of versions that didn't exhibit any bugs that bite them, but they're precluded from doing that.

There are always scenarios with problems in which code depends on other packages. I disagree with your argument, but if no-one else agrees with me I'm fine. Gary

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Benji York wrote:

On Mon, Dec 8, 2008 at 9:18 AM, Gary Poster <gary.poster@gmail.com> wrote:

...

On Dec 8, 2008, at 9:02 AM, Benji York wrote:

...

On Sat, Dec 6, 2008 at 10:28 AM, Christian Zagrodnick <cz@gocept.com> wrote:

...

Log message for revision 93722: - Switched to Zope 3.4 KGS.

- New lines are no longer stripped in XML and HTML code contained in a textarea; requires ClientForm >= 0.2.10 (LP #268139). This revision make the buildout fail with

Error: Couldn't find a distribution for 'ClientForm>=0.2.10'.

I suspect you had that version of ClientForm in your cache and didn't realize that it is not available in the KGS index.

Even if we fixed that, I don't want to require a particular version of ClientForm in testbrowser. There's no need to impose a newer version on people who don't need it. Anyone who does need the bug fix can specify the newer version in their project. FWIW, I disagree. The specification that you removed is exactly the sort of thing that I think is appropriate in setup.py. The tests will now fail (I assume, since I believe Christian Z added testbrowser tests for the failure caused by the ClientForm bug) with a lower version of ClientForm, so it is appropriate to set the value in setup.py.

Nope, the tests will pass in the zope.testbrowser buildout.

Having tests which pass only in that buildout is an "attractive nuisance": I'm seeing test failures in the version of zope.testbrowser linked into the Zope2 trunk, for instance. If the behavior of zope.testbrowser is broken in the absence of a sufficiently-recent version of ClientForm, then that behavior should be spelled out in setup.py's dependencies: that is what they are for.

However, if a testbrowser user for some reason run the testbrowser tests outside of its buildout, then you're right, they may see a failure if their versions aren't new enough. At that point I would hope they would read the CHANGES.txt and see that a new version is required.

The trade off is in requiring people to upgrade a dependency in order to get a bug fix that they may not care about.

In the large, this is the problem with specifying versions in setup.py. Doing so assumes too much about how people are using all the dependencies involved.

Here's a scenario: I fix a bug in some package, it depends on a newer version of say, zope.publisher. I put that requirement in my package's setup.py. A user of my package upgrades to get an unrelated bug fix and is forced to use the newer zope.publisher. It so happens that that the new version of zope.publisher has a different bug that bites them. They now are in a bad spot.

A user of your package cannot rely on automatic dependency resolution in this case: your package is broken without the new version of its own dependency, so they can't upgrade to it. Stripping all versions from the dependencies in setup.py only works if users are willing to run their own package indexes, and figure out edge cases such as the ones you describe by themselves: at that point, forking the egg to drop the manually-resolved extra dependency is a minor nuisance.

If the setup.py hadn't specified a version then the programmer could have constructed a set of versions that didn't exhibit any bugs that bite them, but they're precluded from doing that.

Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJPVgV+gerLs4ltQ4RAof1AJ49K2CxxaJv18rm1U9rHtRjLxLHbQCdGAVu xjQLdJ5ceRek83aNa6R3fag= =tFoW -----END PGP SIGNATURE-----

On 2008-12-08 18:23:33 +0100, Tres Seaver <tseaver@palladion.com> said:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Benji York wrote:

...

On Mon, Dec 8, 2008 at 9:18 AM, Gary Poster <gary.poster@gmail.com> wrote:

...

On Dec 8, 2008, at 9:02 AM, Benji York wrote:

...

On Sat, Dec 6, 2008 at 10:28 AM, Christian Zagrodnick <cz@gocept.com> wrote:

...

Log message for revision 93722: - Switched to Zope 3.4 KGS.

- New lines are no longer stripped in XML and HTML code contained in a textarea; requires ClientForm >= 0.2.10 (LP #268139). This revision make the buildout fail with

Error: Couldn't find a distribution for 'ClientForm>=0.2.10'.

I suspect you had that version of ClientForm in your cache and didn't realize that it is not available in the KGS index.

Even if we fixed that, I don't want to require a particular version of ClientForm in testbrowser. There's no need to impose a newer version on people who don't need it. Anyone who does need the bug fix can specify the newer version in their project. FWIW, I disagree. The specification that you removed is exactly the sort of thing that I think is appropriate in setup.py. The tests will now fail (I assume, since I believe Christian Z added testbrowser tests for the failure caused by the ClientForm bug) with a lower version of ClientForm, so it is appropriate to set the value in setup.py.

Nope, the tests will pass in the zope.testbrowser buildout.

Having tests which pass only in that buildout is an "attractive nuisance": I'm seeing test failures in the version of zope.testbrowser linked into the Zope2 trunk, for instance.

If the behavior of zope.testbrowser is broken in the absence of a sufficiently-recent version of ClientForm, then that behavior should be spelled out in setup.py's dependencies: that is what they are for.

That's the way I think of it, too. Also the bug introduced by an older ClientForm is so subtle that it is not immediately obvious that it is the testbrowser which is broken. It may if you just got an upgrade, but not if you setup your things and it just behaves strange.

...

However, if a testbrowser user for some reason run the testbrowser tests outside of its buildout, then you're right, they may see a failure if their versions aren't new enough. At that point I would hope they would read the CHANGES.txt and see that a new version is required.

The trade off is in requiring people to upgrade a dependency in order to get a bug fix that they may not care about.

In the large, this is the problem with specifying versions in setup.py. Doing so assumes too much about how people are using all the dependencies involved.

Here's a scenario: I fix a bug in some package, it depends on a newer version of say, zope.publisher. I put that requirement in my package's setup.py. A user of my package upgrades to get an unrelated bug fix and is forced to use the newer zope.publisher. It so happens that that the new version of zope.publisher has a different bug that bites them. They now are in a bad spot.

A user of your package cannot rely on automatic dependency resolution in this case: your package is broken without the new version of its own dependency, so they can't upgrade to it.

Stripping all versions from the dependencies in setup.py only works if users are willing to run their own package indexes, and figure out edge cases such as the ones you describe by themselves: at that point, forking the egg to drop the manually-resolved extra dependency is a minor nuisance.

Agreed. -- Christian Zagrodnick · cz@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 4 · fax +49 345 1229889 1 Zope and Plone consulting and development

On Dec 9, 2008, at 8:43 AM, Benji York wrote:

On Mon, Dec 8, 2008 at 12:23 PM, Tres Seaver <tseaver@palladion.com> wrote:

...

Stripping all versions from the dependencies in setup.py only works if users are willing to run their own package indexes, and figure out edge cases such as the ones you describe by themselves:

The above claim appears false. Every project (both the open and closed) I work on has virtually no versions in setup.py and we don't use a private package index. We use a version section in the buildout.

The "virtually" is the catch here. They do have some. They are typically introduced when an older version of a dependency *does not work with the software*. To me, "does not work" == "breaks tests". Because our community, and others, try for backwards compatibility, this kind of assertion happens relatively rarely. These setup.py assertions are always, or almost always, not "pinning" but saying "I work with X or better." I'm advocating these sorts of "X or better" assertions. Christian's zope.testbrowser change fits these characteristics. Do you, or Stephan, or anyone else with your opinions, have some other additional line that must be crossed, or do you assert that setup.py should never have any version numbers? For instance, if you have a project that requires a newer API in, say, zope.component, do you assert that it is inappropriate to specify a zope.component of "X or better" in your setup.py for that project? Gary

On Tue, Dec 9, 2008 at 9:01 AM, Gary Poster <gary.poster@gmail.com> wrote:

On Dec 9, 2008, at 8:43 AM, Benji York wrote:

...

On Mon, Dec 8, 2008 at 12:23 PM, Tres Seaver <tseaver@palladion.com> wrote:

...

Stripping all versions from the dependencies in setup.py only works if users are willing to run their own package indexes, and figure out edge cases such as the ones you describe by themselves:

The above claim appears false. Every project (both the open and closed) I work on has virtually no versions in setup.py and we don't use a private package index. We use a version section in the buildout.

The "virtually" is the catch here. They do have some.

Yep. I maintain that we'd (or at least I) would be better off with fewer (but quite possibly not none, see below).

They are typically introduced when an older version of a dependency *does not work with the software*. To me, "does not work" == "breaks tests". Because our community, and others, try for backwards compatibility, this kind of assertion happens relatively rarely.

These setup.py assertions are always, or almost always, not "pinning" but saying "I work with X or better." I'm advocating these sorts of "X or better" assertions.

The problem arises when your definition of "works" (the tests pass) conflicts with another user of the package's idea of "works". Individual packages do many things for all those things to work simultaneously, then yes, the tests should pass. However, some users of that package may not need (or want) all of the things that a package does. Let me try an example. Lets say that a new version of a package (A) that I use comes out and it has some new features. Some features I would like to use, others I'm not interested in. Upon trying the new version setuptools/buildout complains that there is a version conflict because there are minimums in package A's setup.py. The new package's declared minimum for package B is higher than what I'm using, so I upgrade package B. After doing so I run my tests and find that they fail. After investigating, I find that the new version of package B has changed behavior or introduced a bug such that it is unusable for me. However, I find that if I use the previous version of package B my tests pass and the new features of package A that I want also work, then I'm happy; even though package A is "broken" it still works for me. I now have time to track down the problem(s) in packages A and B and reconsile them with their author(s) while still benefiting from the new features I need in package A. Of course, since the versions are specified in setup.py, I have to fork project B, so that's a little irritating. Let me re-run that scenario with no versions in setup.py: Upon trying the new version setuptools/buildout does not complain that there is a version conflict. I run my tests and find that they pass. I declare my upgrade a success and go on about my life. Someone might come back with a rejoinder that when I want to use the features of package A that don't work with the version of package B that I'm using I'll have a problem. True. Is this better or worse than the alternative? I think it's better because I only deal with version issues if I must, not just because version minimums are forced on me. I'm sure someone with a better imagination can come up with additional/better examples.

Christian's zope.testbrowser change fits these characteristics. Do you, or Stephan, or anyone else with your opinions, have some other additional line that must be crossed, or do you assert that setup.py should never have any version numbers?

The only time I can imagine a version number in a setup.py being a good idea is if a lower version of the dependency will have truly detrimental affects (like it has been turned into a trojan) or if every conceivable use of a package is broken without that minimum version. But even "being able to import an interface" is a conceivable -- and common -- use of a package, and that works no matter what version of dependencies are used (normally).

For instance, if you have a project that requires a newer API in, say, zope.component, do you assert that it is inappropriate to specify a zope.component of "X or better" in your setup.py for that project?

That sounds like something that should be documented in the CHANGES for the project, but I don't think forcing the version number is warranted even in that case (e.g. "I just want to import an interface why do I have to upgrade a dependency"). I wonder if the best possible world might be one in which setup.py versions are overridable by buildout's version section. That would seem to give people both options. In fact, that sounds really appealing to me. I'd like to know that a package author thinks I should use a newer version, while maintaining the option of easily ignoring their advice. I am still a little confused though. I guess people who want minimums in setup.py don't nail versions in their buildouts. (?) This message brought to you by -vvv. ;) -- Benji York Senior Software Engineer Zope Corporation