We're going to be going ahead with the repository re-migration, to a symlinks based scheme, this afternoon. There will be some downtime, when checkouts are unavailable, and then existing checkouts will have to be redone. I'm aiming for 3:00 PM Eastern Standard Time (0800 GMT), and expect the migration to take less than two hours - done by 5:00 PM (1000 GMT). I'll post another notice about a half hour before i shut the pserver down. Ken Manheimer klm@digicool.com On Wed, 1 Aug 2001, Ken Manheimer wrote:

We've hit on some snags in the process of using the new CVS arrangement. The organization is good, but the use of CVS modules turns out to be deficient - enough so that we're evaluating a different scheme to implement the organization. If we shift to the new scheme, it will mean having to once again do new checkouts - old checkouts will not update.

We'd like to scope this out and make the switchover quickly, to reduce that amount of time that people get settled into the old new scheme. I figure i'll have evaluated the new scheme by the end of the day, and (presuming it's ok) will allow another day for word to filter out (and to prepare things for the shift), implementing the shift on friday, Aug 3.

In the interest of getting CVS-savvy attention to the issues and plans, i'm sketching out the details of the situation, below. For those of you that aren't really interested, the important thing is that you'll have to (once again) redo your checkouts when the changeover happens, probably this friday.

The Problems

The first problem is that structural changes to a module-based recipe are not tracked by CVS updates. If someone adds or deletes elements being composed, people with existing checkouts do not see them unless they redo the checkout.

It turns out that people can redo the checkout on top of their existing checkout, and it'll preserve local changes, but this is not satisfactory, for a few reasons:

- It requires that updates be of the entire checkout, without the option to do just sections.

- These "updates" have to include all the information of the original checkout - you have to designate the server:host:dir each time.

It's just untenable.

The other problem with modules is that they're invisible to our web interface to the repository, ViewCVS (and i expect the same holds for alternatives, like cvsweb). This is a serious drawback for those who depend on the web views for discovering what's there, or for tracking or reporting changes.

The Proposed Solution

Instead of using modules, we will use symlinks to knit together the composite applications.

CVS works with respect to the files as they're presented by the filesystem. As far as CVS is concerned, a symlink is the file to which it points. By using doing the composition with symlinks, addition or deletions of links is treated just like addition or deletion of the targeted directories, so CVS updates will properly reflect the changes. We choose symlinks instead of hard links to be explicit about the sharing relationships - to indicate where the primary location is, as opposed to the places that share access to the primary, using symlinks.

ViewCVS also treats symlinks like files/directories, so it can be used to navigate the symlink-based recipes.

There are some relatively minor drawbacks to the symlink-based approach, which i describe below, mostly to document them for future reference. I believe they're way outweighed by the benefits.

Below is the list of drawbacks and benefits. I'm going to test out the symlink-based scheme soon, and if it proves satisfactory (and barring objections), will be aiming to implement the switchover on friday, Aug 3.

Benefits:

- As with modules, we can organize the repository in sensible ways for sharing packages, products, and so forth, in multiple applications. We can transparently change actual repository file locations at will, moving the symlink targets to track the changes.

- Unlike modules, indirected additions and deletions will be inherently tracked by CVS updates.

- The composed elements will be visible to tools like ViewCVS as explicit elements, rather than invisible like they are now.

Disadvantages:

- Changing these symlink-based repository structures currently requires repository host access. We don't want to give that out, so that'll impose going through an administrative bottleneck for people wanting to make changes.

We plan to provide a simple administrative (CVSROOT) file which dictates the crosslinking structure, and a mechanism which effects the changes when the file is checked in. Until then, people without authority (almost everyone) will have to go through administrators to get such changes implemented.

- Modules enable a bit more recipe reuse. For instance, if you wanted two different versions of Zope, one with CMF included and one without, you could reuse a core Zope module in a ZopePlusCMF module. We will be able to compensate, perhaps better, with distribution front-ends.