[Zope3-Users] need guidance

[Chris McDonough]

On Nov 30, 2005, at 12:38 PM, Brad Allen wrote:

> At 1:17 AM -0500 11/30/05, Chris McDonough wrote:
>> While you're developing, it's much easier because the *content* in  
>> your ZODB doesn't typically have any value during development.  If  
>> it does have value during development, you have a development  
>> model that's not similar to mine.
>
> Well, I generally try to follow an iterative approach like that  
> described in Agile or Extreme Programming styles (<http:// 
> en.wikipedia.org/wiki/Extreme_Programming > ). The idea is to get  
> something into production very quickly, let users bang on it, and  
> then produce a new version based on feedback from users...and then  
> do it again, and again, etc. There is no big lead time for  
> "development-only"; the cycle is more like develop-release-use- 
> develop-release-use, etc. During that cycle, end user data needs to  
> be carried over to each new version.

FWIW, I wouldn't characterize the system into which those users put  
content into as a development system (aka development sandbox).   
Instead I'd call it a "demo" sandbox.  Sure, the data in a demo  
sandbox needs to be enduring or you can't do iterative development  
because the customer will have no frame of reference between  
iterations, as all of their content will be gone.

But data in a *development* sandbox is not very interesting and can  
be changed or thrown out at will.  For example, I really don't care  
what's in a news article on my development sandbox.  If the title of  
a news article says "Bush Invades Grenada" or "Ipsum Dolorum", it's  
all the same to me.  I'm not actually reading the content, I'm just  
making it possible for the customer to add and change content  
abstractly.  What the content says has no value to me.

But there is the issue of keeping the demo system up to date while I  
do development because it *does* have value to the customer, so it's  
of course something that needs to be dealt with.

So to get there, the mechanics I use are like this:

- I create a buildout that builds the software I need to create a  
sandbox.
   Typically, this is Zope, ZEO, maybe a relational database, maybe  
an LDAP
   database, and so forth.  I've done this many times, so it typically
   only takes me half a day or so, because I steal judiciously
   from other buildouts I've done.  The job of the buildout is to put  
files
   on the filesystem.

- I create a site generator, that puts some data I need into the ZODB.
   The job of the site generator is to put data in the ZODB.  This also
   takes just a little bit, typically, because I've done it many times.

- As I do development, I create scripts which evolve the database  
content
   from one code generation to the next as necessary.

When I want to set up a new development sandbox, I run the buildout  
and the site generator on my own system.  Then I continue to write code.

When I finish "iteration 1", I set up a demo sandbox for the customer  
on one of his systems using the same buildout and site generator  
mechanisms.

I continue developing in my development sandbox on my own system.

In the meantime, presumably my customer is using and evaluating the  
"demo" system, which is running a only-slightly-older version of the  
code than what exists in my development sandbox, and also contains  
"real" data due to the fact they're adding and changing content.

The customer never sees my development sandbox, only I do.  I don't  
have his new content, but I don't really care.  I'm able to deal more  
abstractly with the problem we're trying to solve than he or she  
might be, so having the most up to date content (or sometimes not  
having any content at all) in my development sandbox is not really an  
issue for me.

In the meantime, if I'm working with other developers, they also run  
the buildout and site generator code and work in a sandbox *on their  
own system*.  They never see page renderings generated by my  
development sandbox.  I never see theirs.  When we need to share  
code, we do so by checking our Zope product code in and out of CVS or  
subversion.  We maintain the site generator and buildout as necessary  
as things change.

As we move towards another iteration, if I need to, while I'm writing  
code, I write evolution scripts that are capable of fixing ZODB data  
in place between milestone revisions of code.  I happen to know  
exactly when I need to do this, because I've been working with ZODB  
for many years.  Other developers working with me do the same.

When the project reaches a development milestone, I push out my  
development code to the customer's demo system and run the evolution  
scripts on the demo system.  Their content remains intact, and they  
begin evaluating again and we start all over again for the next  
milestone.

You say above that there's no big lead time for "development only".   
If you mean that you can't make time to do the above, I can  
understand the sentiment; I've heard it many times.  But I just  
haven't found any other sane way to do it over the years I've been  
using Zope, so I feel like I've learned how to do this "the hard way"  
and I think it has a lot of value; every other method I've tried  
falls down in one or more important ways, particularly when I need to  
do collaborative development.

HTH,

- C