[Zope-dev] Reflections on the Zope 2 to Zope 3 transition

[Jim Fulton]

Philipp von Weitershausen wrote:
> Jim Fulton wrote:
> 
>>> 2. Especially Andreas expressed his worries about the current release 
>>> policy in Zope 2 and its future regarding maintainance and support. I 
>>> have to say that I share some of his skepticism regarding Zope 2. I 
>>> personally have never fully understood ZC's reasons for the release 
>>> roadmap as it is. I might not see the big picture, but I know I would 
>>> have done it differently. I've always tried to make that clear in the 
>>> past.
>>
>>
>> I'm surprised to read this. Could you be more specific about your 
>> concerns?
> 
> 
> I should have said "the release roadmap as it WAS". I was very skeptical 
> about the original plans to make Zope3 backward compatible. I am still 
> very skeptical about the ability to add conversion scripts. They would 
> be incredibly tedious painful to write and I personally would rather 
> migrate code manually than trust a script. After all, the paradigms have 
> changed a lot.
> 
> I see it as realistic as Stephan. There is no sane migration to Zope3 
> than the one of refactoring code. Now, in order for that to happen, we 
> need the CA in Zope2, so people can start asap. My only criticism has 
> been and still is that the CA hasn't been introduced to Zope2 earlier. 
> Now, I know that a) the CA has been refactored a lot since its invention 
> (and IMO only for the good) and b) there was a lack of resources to do 
> that actual integration. That's why I've never declared anyone guilty 
> for it (which is why you may be surprised by this).
> 
> It sure would have been nice if Gary had shared his experience with 
> FrankenZope a little earlier. I remember Martijn being quite eager to 
> experiment with it, too. But that's the only constructive criticism I have.

So your concerns are really about the Z2 to Z3 transition plan, not about
the Zope release process.

Before I get into the main topic of my response, I'll note that I'm a
good bit more optimistic about backward compatability than you are.
I just have an intuition that we'll be able to do much more than you
or Stephan expect. It's an intuition, not a promise. I can't prove it.
Only time will tell. :)

I know that lots of people are concerned by the uncertainlty about the
future transition.  This is a tough situation.  I made a number of
tradeoffs that put us in this situtaion.  I think they were the right
tradeoffs and I'd make them again.  But, as with any tradeoff, we've
had to balence positives and negatives.  Let me explain:

Zope 3 has always been a relatively small project.  Within Zope
Corporation, I'm the only one that has worked on Zope 3 full time
for any length of time.  As CTO, even my time on Zope 3 is
not truly full time, but it's closer than for anyone else at ZC.
Customer work (Zope 2), important community issues (like this thread,
or like the security issues we uncovered last fall) take precedent.

Other people working on Zope 3 have "day" jobs.  They have to go to
school or or do customer work (usually in Zope 2) to make a living.
(Of course, you know this Philipp. :)

I'm not complaining. This is reality and a reality we planned for.

I made two choices that were controversial:

1. I decided not to be encumbered by backward compatability.  This was
    refactoring mercelessly" applied in the extreme. This had the result
    that, except in a few areas, we did start from scratch.

    Pros: We had freedom to think abstracty about how Zope should work, and how
          we could make the experience for the developer as good as possible.
          The premise of not being backward compatible with Zope 2, also
          made us realize that we didn't have to be backward compatible with
          our own Zope 3 work.  We were free to try things out, see how they worked
          and often, totally change them to make them better.  Many core parts
          of Zope 3 have been redone, some more than once. This wouldn't have been
          possible if we had been trying to be backward compatible.

    Cons: We're not backward compatible.  Will we ever be? I'm confident that we
          will have a decent transition.  I don't know what shape that will take
          but I am still confident.  Will there be bumps along the way? Definately.
          The recent issue with the conflicts between the "Zope" and "zope" package
          is a good example.

    While many might not agree with me, I am glad I made this decision. I would
    make it again. It was the right decision for Zope.

    If we had a lot more resources, we might have been able to pay more attention
    to Zope 2 compatability, although I'm not sure that that would have been the right thing
    to do.

2. I decided not to think much about the transition until Zope 3 has gotten farther along.
    The transition is a road from where we are to where we're going.  It's awfully hard
    to build a road when the destination is changing.  This means that, as the Zope Pope
    and CTO of Zope Corporation, I can promise that the road will be built, but I can't tell
    you what precise route the road will take or what it will be like.  I can only give
    a general direction.  Frankly, the destination has been well enough defined for a while
    now that we could begin defining the transition, but we just don't have enough resources
    to do so.

    Pros: We didn't waste time planning transition to a moving target. We ere able to
          focus scarce resources on getting a Zope 3 baseline.

    Cons: Fear, uncertainty and doubt

    Do I regret the decision not to define the transition better sooner? No.  Not given
    available resources.  I know that this puts the community in a tough spot, but I
    don't know what else I can do.  Fortunately, I think we're pretty close to a time
    where we can provide a lot more attention to the transition and I think that much more
    clarity will emerge over coming months.

My main regret is so badly underestimating how long it would take to reach where we are.
There are a lot of reasons for being so far off.  One important reason is that we
created a different culture, one that is far more quality driven.  We are being very picky
about what we put in the Zope 3 baseline.  We're willing to take the time to get it right,
or at least as right as we can make it.  We couldn't do this if it wasn't for Zope 2.
Ultimately, I think that this cultural shift will provide us with great benefits.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org