[Zope3-Users] Eggs and classic instance homes, again... (was Re: Eggs, workingenv.py, and 'classic' instance homes)

[Darryl Cousins]

Hi Jeff,

I hope that this might at least give you the optimism you appear to be
lacking. 

I've been migrating a complex project which was running on Zope-3.2
tarball install.

My first step was to get it all into a buildout. buildout.cfg looked
something like this [1]. I've missed out lots here (postgresql, lxml,
sqlos, sqlobject, psycopg2, pyro, zif.jsonserver amongst others). Note
that I have included a local install of python in parts to keep things
clean and tidy.

Second step was to upgrade to Zope-3.3 - this just meant changing the
url in the zope3 section. An hour or two of fixing deprecation warnings
(really only deprecation stuff, nothing major).

Step Three, where I'm at now, is to change the buildout to use Zope-3.4
eggs, and Paste. Zopeproject buildout.cfg and other files could provide
you a starting point. I'm using
http://grok.zope.org/releaseinfo/grok-0.10.2.cfg to pin down versions.
See http://grok.zope.org/releaseinfo/readme.html for info on using this
file.

I haven't got the server running yet, but then I've only just finished
getting buildout.cfg right, and all parts built.

In all there has been several hours work so far, but that includes
sorting out the additional dependencies (postgresql etc). Before the
buildout the whole application was a nightmare to install. Now its
looking sooo much better.

I would skip step two if I did this again, although I imagine there will
be a lot more import and deprecation errors to correct.

Hope that helps.

Regards,
Darryl

[1] buildout.cfg

[buildout]
index = http://download.zope.org/ppix
develop = .
parts = zope3
	server
        instance
        database
        test
        projectpackage # the application code
find-links = http://download.zope.org/distribution/
download-cache = cache
eggs-directory = eggs # keeping this install fully separate from others
develop-eggs-directory = develop-eggs

[zope3]
recipe = zc.recipe.cmmi
url = http://www.zope.org/Products/Zope3/3.2.0/Zope-3.2.0.tgz
extra_options = --with-python=${python:location}/bin/python 
                --force

[server]
recipe = zc.recipe.egg
eggs = ${projectpackage:eggs}

[instance]
recipe = zc.zope3recipes:instance
application = projectpackage
zope.conf = ${database:zconfig}

[database]
recipe = zc.recipe.filestorage

[test]
recipe = zc.recipe.testrunner
eggs = ${projectpackage:eggs}
extra-paths = ${zope3:location}/lib/python
working-directory = ${buildout:directory}/test
defaults = ['--tests-pattern', '^f?tests$',
	'-v'
	]

[projectpackage]
recipe = zc.zope3recipes:app
eggs = ${interpreter:eggs}
site.zcml = <include package="projectpacakge" /> # all zcml defined here

[python]
recipe = zc.recipe.cmmi
url = http://www.python.org/ftp/python/2.4.4/Python-2.4.4.tar.bz2
executable = ${buildout:parts-directory}/python/bin/python
extra_options = --with-threads
                --with-readline
                --enable-unicode=ucs2

[interpreter]
recipe = zc.recipe.egg
eggs = setuptools
       # list-of-eggs: lxml, sqlos etc, etc
interpreter = python
extra-paths = ${buildout:parts-directory}/zope3/lib/python




On Wed, 2007-11-07 at 17:24 -0700, Jeff Shell wrote:
> The last time I was active on this list was back in mid-august. I was
> fighting an uphill battle to try to figure out how to migrate our
> software to 'Eggs'. I ultimately had to abandon the process as there
> was just too much real work to be done, and I was getting conflicting
> answers about how to move forward. Actually, I couldn't get much help
> from the community at all. I apologize for the crankiness of this
> email. The last time I tried to get help on this topic was an
> excercise in sheer frustration. Since then, the gulf I've worried
> about seems to have gotten even bigger, and I'm feeling very stressed
> right now.
> 
> So now I see all that's going on with Zope 3 and I still don't
> understand a thing about how to move forward. We have about twelve
> active customers deployed on Zope 3, on top of one or two large
> frameworks that we've built on top if Z3. We still use the basic
> 'instance home' layout. We don't use eggs - we just check things out
> directly into the instance home. Even stuff from the Zope community,
> because it's easier to do that than to even get workingenv or any of
> these other things running.
> 
> I'm still lost about what to do. Grok is of no use to us at the
> moment. Many of these apps have been up and running for longer than
> Grok's been alive. Is there any kind of migration documentation out
> there? Or have these new ways of doing things been in use for so long
> that most people have just rolled along with them? I just don't have
> the time. I scarcely have time to keep up with the lists (which is
> obvious since I haven't read a thing for nearly three months).
> 
> I'm going to resume my conversation here, responding to myself.
> 
> August 16, 2007, Jeff Shell wrote:
> > > Jeff Shell wrote:
> > > > But now I'd like to be able to install that into "classic" zope 3.3
> > > > instance homes, trying to be as unobtrusive as possible.
> > >
> > > How have you installed other packages before? I suspect by just dumping
> > > their source in INSTANCE/lib/python. Any reason not to continue to do that?
> >
> > Because I feel like we're falling further and further behind, or at
> > least trapped in our own little world.
> 
> Yep, this is becoming even more true. We're now even further behind.
> 
> > But yes, this is how we did it. I ended up writing most of
> > rocketbuild/rockout (our Rake-ish tool) because of this. There are a
> > lot of external tools out there that we still don't use because we
> > don't have the time to deal with the headache (made minor by
> > `rockout`, but still present) of dumping sources into lib/python. For
> > internal stuff, it's not so bad.
> 
> I want to get away from this (using source control as distribution
> mechanism). Eggs and buildout are supposed to help here, but
> ultimately it was easier to roll our own tool as we just could not
> figure out how to apply them to our configurations.
> 
> We're about to do a massive server upgrade, which would be a good time
> to sneak in new deployment practices.
> 
> > The longer we avoid whatever is going on with setuptools and buildout,
> > the harder it will be to migrate. It's hard enough already.
> 
> Yep. Feels even harder now, but probably because they cause me so much
> stress just by looking at them.
> 
> > > That seems backward. If you want to install an package into a "classic"
> > > environment, it doesn't seem that surprising you'll also have to do it
> > > the "classic" way...
> >
> > It doesn't seem entirely backward. I mean, it does. But at the same
> > time... I just don't know how it's all supposed to work. But if we
> > keep doing things in the 'classic' way without trying to step forward,
> > then I fear we'll stay this way forever because it's already scary
> > enough peeking over the hill at how buildout and all of that stuff
> > works and how different it is from how we've done things. I don't want
> > the gulf to get bigger, because I think that with our plans for growth
> > and for a more distributed deployment server setup, 'buildout' will
> > help tremendously.
> >
> > I'm just trying for baby steps right now, because it feels like we've
> > still got a long way to go to get to anything better.
> >
> > I saw mention of 'workingenv' in your book, so I assumed it was or is
> > possible to easily have instance homes with locally managed eggs.
> > Shouldn't it be?
> 
> Still have a long way to go. Still have no answers for what to do or
> how to start migrating. Some of our problem is caused by the stupidity
> of CVS, which will go away when we migrate to Git.
> 
> Is there going to be a Zope 3.4, for real this time? Will it offer the
> clues I'm looking for? 'zopeproject' says 'You can start a new
> Zope-based web application from scratch with just two commands'. But
> what about those of us who have started many many Zope based web
> projects?
> 
> Thanks,
> Jeff Shell
> _______________________________________________
> Zope3-users mailing list
> Zope3-users at zope.org
> http://mail.zope.org/mailman/listinfo/zope3-users