[Zope-dev] Re: buildout 'versions' and 'develop' conflict

[Martijn Faassen]

Christian Theune wrote:
[snip]
> Nope. I'm not always working against a fixed version list. E.g. when I
> developt z3c.zalchemy then this is a library package, not an
> application, so I don't fix the versions but let anything that satisfies
> the the requirements in setup.py come in.

This thread is called 'buildout "versions" and "develop" conflict, and I 
was talking about that topic. I can see how this is related to other 
topics from the perspective of consistency arguments, but perhaps you 
should be more clear. :)

> In this situation I might use a checkout of SQLAlchemy that doesn't
> match the version requirement so it doesn't get picked. (I.e. there are
> two branches of zalchemy, one for SA 0.3 and one for 0.4)

And you're listing both of them in 'develop'?

Fred pointed out to me that you might want to rely on the same package 
but different versions multiple times in a buildout. I can see how this 
can be useful, but I don't see how one can use the 'versions' 
construction of the [buildout] section and do this.

> I don't want to start developing on the 0.3 branch using SA 0.4
> accidentally.

I think that this information should be codified in the setup.py 
requirements of your package. If it relies on 0.3, it should say so with 
a dependency range. I don't think pinned versions are very useful to 
stop you from making accidents here.

> buildout can be and is used in a much broader way than just applications
> that provide fixed versions.

My summary:

The consistency argument indicates to me that version pinning should 
have the same effect as listing version constraints in setup.py. A 
reasonable argument. Since version constraints are the most specific 
constraints possible, if there is no constraint conflict they will be 
the ones picked.

There is a use case argument that allows the listing of multiple 
versions of a development package in 'develop' at the same time, and 
then have different parts of the buildout pick up on it. Sounds like a 
valid use case. I wonder though what will be picked up if you *don't* 
specify the version explicitly later on in your buildout. Unspecified 
behavior?

Having multiple versions of the same package listed in the 'develop' 
section can never work, I take it, in the face of version pinning. If 
you have two versions listed in 'develop' and want to use different ones 
in different sections of your buildout, you can't do this in the face of 
a global version list, right? I'd say the consistency argument implies 
buildout should fail with an error if you try to use two different 
versions of a package while the version is pinned down globally.

The task-driven and most-common use case argument ("I want to develop!") 
have made the 'develop' section trump any other listed packages if the 
version constraints allow both. This is the current behavior.

The task I am concerned about entails wanting to hack on a package that 
is pinned down. This is in my experience very common in the face of 
shared lists of pinned versions (from the KGS or Grok).

In the face of such lists, it's not very pleasant to have to explain to 
people who want to hack on a package that they should also add the 
package to the 'versions' section of a buildout to override any entry of 
them in the list.

The goal of this feature is to allow us to tell people the following:

* if you want to hack on a package, check it out into your buildout and 
add it to 'really_develop', and then re-run bin/buildout.

and that this will work even in the face of explicit version pinning.

My proposal is to add a new feature to buildout, temporarily called 
'really_develop', which places a package on the develop list and trumps 
version pinning.

We can debate whether it should also be trumping any version constraints 
  of other packages as well - I don't know whether this is a good idea 
or not. I think perhaps it should. It *should* still obey the 
constraints listed in the setup.py of the package under development however.

In constraint terms, there would be the ability to override existing 
constraints by placing a package in 'really_develop'. You could see this 
as an 'override' version: a package listed in 'really_develop' gets a 
version 'override' that is guaranteed to fulfill any constraints on 
version *and* is the most-preferred version.

Listing a package in 'develop' doesn't change the version but makes that 
version the most preferred version if multiple versions of that package 
would fulfill the set of constraints.

Regards,

Martijn