[Zope3-Users] Using buildout for Zope 3 development instances (not developing Zope 3, but for developing against it)?

[Jeff Shell]

I started taking another look at zc.buildout last night and am happy
to find that the documentation has improved dramatically from when I
first looked at it. And I'd really like to start using it on our new
projects. But I don't know where to begin, nor how much we (my
company) might have to alter our development or deployment strategies.

I don't know if this is the right list to ask these questions on, but
my questions are specific to using Buildout to automate building
development-focused Zope 3 instances, more than they are about using
Buildout itself.

I would like to maintain our current development setup, where our
local Products are checked out of a repository into
`$INSTANCE_HOME/lib/python`,  while also ensuring that all third-party
dependencies are also installed into that particular instance home.
Keeping track of dependencies has been enough of an annoyance up until
now that we've kept our use of third party products low. We knew there
were good ones out there, but playing the game of "oh crap, I forgot
to get that!" over and over again gets old. :)

Anyways, my questions:

1) Where are the Zope specific recipes, particularly
``zc.recipe.zope3instance`` and ``zc.recipe.filestorage``? They don't
appear to be in the Cheeseshop. I think that there may be a
``downloads.zope.org`` site, or something like that, but with the DNS
issues affecting *.zope.org right now, I haven't been able to find out
if that's really where they are or should be.

2) I have very little experience with Eggs. I've never developed one.
Heck, I've never really developed a proper 'distutils' style package.
I'm used to making my packages / products in CVS as the root package
module itself, never as something with a 'src' subdirectory and a
'setup.py'. Traditionally, distutils hasn't made much sense for Zope
focused products/packages, so I've never really bothered with it. It's
made it easy to do ``cvs co -d cms Products/cms`` into
``$INSTANCE_HOME/lib/python`` and go merrily along my way during
development. I gather that this should be possible with a distutils
style setup, using ``cvs co -d cms Products/cms/src/cms`` instead?

If the targets for most of our products/packages consist of my company
and its customers, should I even bother with making them into Eggs? We
tend to use CVS checkouts on our deployment servers (a policy that I
wish we didn't do, but it does make it easy to fix critical bugs found
in that environment and get them back to the repository).

3) Does anyone have experience with using zc.buildout to make simple
Zope 3 instances for both development and deployment, where the Eggs
are local to the Zope 3 instance (and  Zope knows where to look to get
the local eggs)? The Zope-specific buildout recipes may address some
of this, but as I can't access svn.zope.org at the moment to study the
source, I can't really tell.

I understand (sortof) how Eggs and Ruby Gems work when installing
site-wide packages, but I don't understand how they work as packages
local to an application (like a Zope 3 instance). I'm too used to just
seeing a package like 'sqlalchemy' on my path as a directory, not as
`SQLAlchemy-0.2.8-py24.egg`.

Ultimately, I would really like to have is a setup that has the following:

- Installs dependent third party products, either as eggs or as cvs/subversion
  checkouts/exports. Installs **locally into the instance home**, but perhaps
  in a separate path from `lib/python`. (Zope 2's ZConfig let you specify
  additional items to add to PYTHONPATH, but I don't think I've ever seen this
  in Zope 3. Am I missing something?) This could include things like
  zc.resourcelibrary, PIL, SQLAlchemy, simplejson, etc.

- Checks out **source** for all of the neccesary internal packages into
  `$INSTANCE_HOME/lib/python`.

- Could install/copy/move some scripts from internal packages (whether from a
  repository or egg) into `$INSTANCE_HOME/bin`. We have some code that I
  always run from `debugzope` that I'd like to run via `zopectl run` (now that
  `run` is back for Zope 3).

My interests right now are aimed primarily at building out development
instances so that my coworker and I can up and running quickly without
having to keep lists around of Subversion commands to copy and paste
for checking out specific revisions/tags of third-party products
(SQLAlchemy, zc.catalog, hurry, etc).

I know I may need to write Recipes for some of what I want to do and I
don't mind that. I just would like to keep our development strategy --
develop in an instance home built on a zope 3 release -- working
smoothly while automating all of the dependencies and
site.zcml/configure bits.

-- 
Jeff Shell