On Wed, 2003-06-18 at 13:18, Luca - De Whiskey's - De Vitis wrote:

On Wed, Jun 18, 2003 at 11:23:15AM -0400, Chris McDonough wrote:

...

Actually the scripts in $INSTANCE/bin aren't shell wrappers for things in $PREFIX/bin, they're shell wrappers for thing in $PREFIX/lib/python/Zope/Startup. But yes, they are shell wrappers.

And so they would be copied for all instance installed: they would not probably be that large, but what about symbolik links?

These files needn't be here really. There are two scripts: zopectl and runzope. They are convenience shell scripts that run zopectl.py or runzope.py scripts that live in a software home after setting certain envvars. Each of these shell scripts identifies a unique combination of software home, instance home, config file path, and pythonpath. These can be unique per instance, so using a symlink to a common set of files will not work. A typical zopectl script could be replaced with the following command line: ZOPE_HOME=/opt/Zope27 \ INSTANCE_HOME=/usr/local/zope/instance1 \ PYTHONPATH=/opt/Zope27/lib/python \ /usr/bin/python2.2 \ /opt/Zope27/lib/python/Zope/startup/zopectl.py \ -C /usr/local/zope/instance1/etc/zope.conf The -C tells where the Zope config file is. The config file represents settings for a single Zope instance. One of these config files is created for each Zope instance via 'mkzopeinstance'.

...

If you made a global control script that worked as per your example, how would you identify where the 'default' instance is?

I was not aware of a zopectl in the repository so i builded one in python for Debian. As i can see it's quite more powerful than the one you are going to include: http://packages.debian.org/unstable/admin/zopectl.html I would care about any comment.

I think this is great. The zopectl that ships with Zope now is capable of controlling a single Zope instance. Your system takes on the larger burden of simultaneous multi-instance Zope configuration and control on a single machine. Your code looks quite nice as well. Your system is a perfect example of a value-added frontend for zope. I can see that many people would want to use this kind of framework. I'd love to see people come up with their own Zope mgmt frameworks and use them as a frontend for Zope instance management with minimal involvement from the Zope source tarball installer. Different frameworks will be more or less valuable on different platforms; for example a Windows multi-instance controller might be a GUI app. As far as the current installation regime goes, I'm aiming to keep common-case (single-instance) installation and operation very simple, much like Apache's. If you want to change your Zope controller to work with 2.7, I think you'll either be very happy (or very disappointed, seeing how much work you put in to creating your own config files) to know that Zope 2.7+ instances have their own configuration file which has a syntax patterned after an Apache configuration file; the parsing system uses the ZConfig package (http://www.zope.org/Members/fdrake/zconfig/). I have attached the zope.conf.in file that represents a valid Zope instance configuration file. As an upshot, z2.py no longer exists and the "zopectl" that runs Zope now on the HEAD knows how to read and interpret the config file. All of the settings you've made configurable within, e.g. your 'default.conf' are configurable via this config file. Additionally, ZConfig lets you perform a textual include of external files into a config file; if you've got common settings in one config file, you can customize the settings for an instance in a small instance config file and %include the common config file into it (even via an HTTP GET if necessary). See the zconfig docs for more information. I'd love to see your zopectl system work with Zope 2.7, and though I likely wouldn't want to package it with Zope, I'd love to see it be able to operate as a front end controller for multiple Zope 2.7 instances. It will not work against the current Zope HEAD because the HEAD no longer has z2.py (everything about Zope startup has changed post-2.6). If you want it to support Zope 2.7, I'd be happy to answer any questions about the new installation/startup process. In particular, I think you'd need to change your zopectl conf file. Instead of keeping all configuration options in the zopectl.conf file, each instance section could keep a smaller set of keys. For example: [Zope] Config-File: /path/to/instance/etc/zope.conf [Instance2] Config-File: /path/to/instance2/etc/zope.conf .. etc... The instance config files could %include a common /etc/zope.conf config file with most of the default settings (instead of using a [DEFAULT] section). You could then throw away most of your parsing code in ZopeCTL.py, and instead of running z2.py in your __start method, run the zopectl associated with the software home, pointing it at the proper config file.

...

FWIW, I'd be very opposed to any sort of limitations on where instance homes could be placed to service this global control script's need to find instances. [...] http://dev.zope.org/Wikis/DevSite/Proposals/InstallationAndConfiguration dated Aug 22, 2002 8:09 pm for more background.

It's my opinion that a zope instance can be described trough a configuration file, thus not requiring multiple commands or links but rather a single command capable of using multiple configuration files. If you combine my zopectl script with the patch is proposed for z2.py (http://collector.zope.org/Zope/925), you'll have a good FHS[1] compliant framework for multiple INSTANCE_HOMEs (being still able to place any file where you like).

z2.py is gone in 2.7, alas. But the pidfile location is configurable in Zope 2.7 within the zope.conf file on a per-instance basis (via the pid-filename directive), so this becomes pretty trivial.

...

This sounds good. I'm not so sure about $SKELPREFIX. Maybe we could [...] libraries in the lib directory). This is why they're currently not broken apart.

To say it all there is only one thing i really need: a way to install _all_ documentation files under a separate directory in a way thay cannot overlap. For what concern Debian, the intallation home will still be /usr/lib/zope untill python fully comply FHS[1] (http://python.org/sf/588756). I'll probably move it to /usr/lib/python2.1/site-packages in future, but i'm still not sure.

Support for installing docs to a specific directory is no big deal. I'll add it to the Zope 2.7 TODO. I'm not sure what the Python FHS-noncompliance bug has to do with Zope. If you're worried about .pyc/.pyo bytecode file generation in a nonwritable directory, I wouldn't because the current Zope 2.7 installer compiles bytecode upon install (thus during runtime it will never attempt to compile any bytecode at Zope startup time). I'd advise against installing Zope library files into site-packages unless you put them in a site-packages subdirectory (like site-packages/zope270 or site-packages/zope271). Otherwise there will be no way to run multiple Zope versions on the same machine sanely because different Zope versions have different libraries. And if you install into site-packages/zopeXXX, it's not much different than just installing them into a completely Python-independent directory like /usr/lib/zope because you'd still have to put that directory on the PYTHONPATH. That said, if you always manage upgrades via your package distribution mechanism, and you only allowed one version of Zope to be package-installed on the machine at once, perhaps putting the files in site-packages wouldn't be such a big deal.

...

In the past, my offers to provide upstream support for Debian packaging hasn't been met with much enthusiasm. If I can make it easier for people to package Zope for Debian, I'd love to, but I'd need to have some buyin and support from the current Debian packagers. Otherwise, it ends up being a waste of time on both of our parts as it won't be what's best for either of us.

It's not that easy to co-maintain the Debian package for Zope: we actually lacks the structures to set up work with non Debian Developers. I suppose this problem will be solved soon, but if you like i can start collecting patches for the CVS repository i've already set up for that purpose. Take a look at https://alioth.debian.org/projects/pkg-zope and its CVS repository.

Thank you, I have. I'm afraid you're going to need to change a lot of things to support Zope 2.7. Please let me know if I can help.

...

Debian's packaging and installation strategy for Zope has always been very customized and divergent from the Zope source distribution's packaging and installation strategy. At very worst, it could stay this way.

Debian has choosed to be be strictly compliant to FHS[1], including it in the Debian Packaing Policy. Not complying to Debian Policy is considered a grave bug for any package in the distribution. Current Zope upstream package is not that compliant, so we always have to modify the package. I hope this will change in future.

Servicing the many goals of installation and configuration is very difficult, but I'm pretty confident we can make your life a little easier in the long term given that I understand your requirements. It will also make Zope that much more "debian-friendly" in the long run. If you can provide specific direction on how to make the Zope source tarball installer service your goals better, I'll be happy to try to implement any suggestions given that it makes sense in the larger cross-platform context. Thanks! - C

On Wed, 2003-06-18 at 20:07, Adrian van den Dries wrote:

Agreed; we can have a separate zopectl package that is dedicated to managing instances.

I'm hopeful that Luca agrees.

...

I'd advise against installing Zope library files into site-packages unless you put them in a site-packages subdirectory (like site-packages/zope270 or site-packages/zope271). Otherwise there will be no way to run multiple Zope versions on the same machine sanely because different Zope versions have different libraries.

May I respectfully ask why there is so much concern with such complicated setups? Surely a production environment (which is what any Zope distribution should aim for) will standardise on a software version?

It's often the case that you want to upgrade to a new Zope (or Python) version on a machine and it's desirable that you be able to run both the older version and the newer version on the same machine during the cutover. For this reason (and others), during production rollouts Zope Corporation actually never uses the system-installed Python on a machine nor does it use binary-packaged Zope distributions. Instead, we install a whole separate tree (typically inside of /home/zope) using CVS and various source buildouts that looks like this: bin/ (control scripts) opt/ (software like Zope, Python, Apache, Zope products, etc.) var/ | |--zope/ (zope instance and data files) | |--apache/ (apache instance and data files) We have make-driven software that creates us a tree via a single command by checking out various version of Python, Zope, etc. from CVS and compiling and installing them. Different versions of Zope and Python, etc. can be installed in opt and we use symlinks to manage versioning. We never install any Zope-related files to a particular Python's site-package directory (even the Pythons we install ourselves) because we like the flexibility of being able to use any Python version we like without needing to copy Zope library files over between Python site-package directories. We just use the stock Zope software home blob and munge the PYTHONPATH as necessary. This works out pretty well in practice because we never have any dependency on any system installed software besides C libraries, CVS, make, and the shell. This makes it easy to deploy Zope to many (possibly divergent) systems without thinking too hard about system incompatibilities and what's installed in a particular Python's site-packages directory.

If the concern is because *developers* want to run multiple Zope instances on multiple Zope versions with multiple Python versions, then I would suggest writing an *alternative* mechanism for developers to mangle their PYTHONPATHs and whatever.

It does benefit developers, but I don't think it's solely for the benefit of developers. It seems that most large systems install themselves this way; for instance you can install multiple Apache versions on a single system without regard for them stomping on one another. Same for MySQL, Postgres, etc. It seems to be a fairly common pattern. The fact that we depend on Python doesn't seem to make much of a difference from the perspective of someone installing Zope. If Apache depended on Perl, wouldn't necessarily expect my Apache to stop working because I deleted some files from my Perl library directory.

...

That said, if you always manage upgrades via your package distribution mechanism, and you only allowed one version of Zope to be package-installed on the machine at once, perhaps putting the files in site-packages wouldn't be such a big deal.

This is doubtless the most common scenario (or it should be).

It is when things go right, but when they go wrong, it's fairly limiting to only be able to have a single version of Zope installed on your system. That said, it's really a packaging decision. I'm currently most concerned about making Zope to be easy to install from source. If I can add features to the source install to make it easy for binary packagers to package Zope, I'll do so, but I won't limit the source installer to installing to a single predeclared place. The current Zope source tarball install will continue to allow you to install multiple Zope software homes on your system without regard to the Python you use to invoke them. The binary packaging scheme for a particular distribution is up to the packager. We can probably provide support in the setup.py file for installing libraries to site-packages to help binary distributions do so if they choose. I don't think it's a particularly good idea, but I'm not in charge of those distributions either, so my opinion shouldn't count for much. ;-)

...

I'm afraid you're going to need to change a lot of things to support Zope 2.7. Please let me know if I can help.

Should we move this to debian-python?

Just tell me where to go and I'll go there. ;-)

...

Servicing the many goals of installation and configuration is very difficult, but I'm pretty confident we can make your life a little easier in the long term given that I understand your requirements. It will also make Zope that much more "debian-friendly" in the long run.

As long as we can distinguish those parts of this exercise that will apply to the Zope distribution from those that should be specific to Debian's package, this shouldn't be too hard.

Yup. Thanks for the input! - C

On June 19, Adrian van den Dries wrote:

Distribution install - most common:: ... Multiple pythons, zopes, instances - most complex::

Note that these aren't mutually exclusive: the porpoise of /usr/local is to allow a site-local hierarchy. That is, the *system* only has one version of the software (and the *system* should only *ever* have one version of the software) and locally-installed packages go in /usr/local, per FHS. The decision to install multiple versions of a package is always a site-local decision, and therefore correctly belongs in /usr/local. I suppose there may be a religious debate a-brewing about whether Zope should install as a Python package into /usr/lib/pythonX.Y/site-packages, or its own package with its own PYTHONPATH. My argument in this debate goes a little like this: A *library* should be installed as a library. Most of Zope's source code is libraries. Zope should install these libraries (packages) according to the regime dictated by the installation of Python that will be used to run Zope. That is, if you are using a distribution-supplied Python to run Zope, you should install into that Python's --prefix. If you are using a custom Python, use whatever regime matches that Python. But remember these are all *customisations*; the defaults all point to the simplest scenario presented earlier. a. -- Adrian van den Dries avdd@flow.com.au Development team www.dev.flow.com.au FLOW Communications Pty. Ltd. www.flow.com.au

On June 19, Chris McDonough wrote:

On Wed, 2003-06-18 at 23:13, Adrian van den Dries wrote:

...

python2.1 configure.py --prefix=/opt/zope271-python21

(FWIW, configure is a shell script.)

Yes, I knew that. I used the .py extension for all the scripts to be consistent, according to my earlier point to this end. It too is just a shell wrapper to find the correct python and palm off the remaining options to inst/configure.py. Why not avoid that altogether and let the user supply the correct python? Ie, instead of:: ./configure --with-python=/usr/bin/python3.2 --other --options you call:: /usr/bin/python3.2 configure.py --other --options Which it pretty much does, anyway. I also see things like this: # Up to six acceptable python versions are allowed. Why six? Where is it enforced?

Are you implying that the stuff that currently goes into software_home/lib/python would be installed to the site-package directory of the Python that was used to invoke the installer? If so, the subsequent invocations above of:

No, I'm saying that the systems should be allowed (or even encouraged) to do this. We already have sys.path and namespaces. In a 1:1 python:zope environment, this should be enough to identify the Zope packages. Hence installing into python's module path. For multiple-version environments, you need --home (see below).

cd Zope-2.7.1 python2.1 configure.py --prefix=/opt/zope271-python21 make; make install

cd ../Zope-2.7.2 python2.1 configure.py --prefix=/opt/zope272-python21 make; make install

... would cause the Zope 2.7.1 libraries to be stomped by the Zope 2.7.2 libraries.

No, because --prefix just told configure.py to install *everything* in that location. If you want to alias this option to --home, that would probably make this clearer, but I don't think it's different in function.

If you mean that the libraries are installed into /opt/zope27X-python21/lib/python, there's no problem and nothing about the installer needs to be changed.

Aye.

So, in any case, given that the ZC source tarball installer will not attempt to manage multiple instances (we'll leave that to Luca)

Aye.

here are the requirements I've gathered so far:

- Add a --doc flag to configure - Add a --skel flag to configure (is this a common pattern?)

I think so. But consider using the Make variables suggested earlier, so that distributions may customise the make step: ./configure --options --supplied --by --zope make SITE=local SPECIFIC=options

- Make it possible to install Zope libraries into the site-packages directory of the Python that invokes Zope's setup.py instead of into software_home/lib/python.

As long as each of the top-level Zope directories in the --home (bin, doc, lib, skel) can be individually tweaked, either in the configure.py or the makefile, I think we win both ways.

Am I missing anything?

Apart from my personal desire to see the shell scripts deprecated, nope. ;-) I think that's covered everything that concerns Zope itself, and the rest gets palmed off to zopectl and packagers. a. -- Adrian van den Dries avdd@flow.com.au Development team www.dev.flow.com.au FLOW Communications Pty. Ltd. www.flow.com.au

On June 19, Chris McDonough wrote:

On Thu, 2003-06-19 at 20:24, Adrian van den Dries wrote:

...

Why not avoid that altogether and let the user supply the correct python?

This is somewhat of a style choice

OK, ZC's call. ;-)

Well, as we all know, shell scripting kinda blows. There is no way that I know of to portably use an array in shell, and I wanted to eventually make it possible to use something other than bash to run the configure script (bash has arrays, but I'm not sure if bourne shell does).

You may be interested in Kenneth Almquist's ash (aka dash in Debian): Description: The Debian Almquist Shell "dash" is a POSIX compliant shell that is much smaller than "bash". We take advantage of that by making it the shell on the installation root floppy, where space is at a premium. . It can be usefully installed as /bin/sh (because it executes scripts somewhat faster than "bash"), or as the default shell either of root or of a second user with a userid of 0 (because it depends on fewer libraries, and is therefore less likely to be affected by an upgrade problem or a disk failure). It is also useful for checking that a script uses only POSIX syntax.

You're suggesting that "--prefix" be changed to "--home". ... You're suggesting that specifying a "--home" implies that libraries will be installed to $home/lib/python.

This is where I get a little confused. How does a user signify that Zope libraries should get installed into the Python site-package directory? What happens if "--home" is left unspecified? Is the "--prefix" option still around? If so, what does it do? Can you give an example of a configure command that would imply that Zope libraries get installed to site-packages?

Erm, yes. I originally wanted --home and --prefix aliased, but I've changed my mind. It should mirror the behaviour of the same options to the distutils' install target. With the distutils, ``--home`` is version-agnostic (installs package ``foo`` into ``$HOME/lib/python/foo/``) and ``--prefix`` is version-aware (installs into ``$PREFIX/lib/pythonX.Y/site-packages``). ``inst/Makefile.in`` could probably be updated to do a --prefix install if $(PREFIX) is defined, else it defaults to a --home install. (Mockups) # the next commands are all equivalent # and produce the install target # ``python setup.py --home=/usr/local/zope`` python inst/configure.py --home=/usr/local/zope ./configure --home=/usr/local/zope ./configure # the next command produces the install target # ``python setup.py --prefix=/usr/local`` # which installs packages to /usr/local/lib/python2.2/site-packages python inst/configure.py --prefix=/usr/local

I think it is possible to tweak many things now via make -e after examination of the makefile.

OK, that is what the Deb should do.

Well, if you ignore the shell scripts, I think we're ok, then!

Peaches! ... (Couple of hours later). OK, attached is a patch to inst/Makefile.in and inst/configure.py that distinguishes --home and --prefix parameters and passes them onto the distutils. But for some reason, there is still no difference between --prefix and --home, even though it's passing those options to the distutils. I suspect it might have something to do with this exerpt from setup.py:: # We create a custom "install scheme" that works the same way on all # platforms. We do this in order to prevent distutils from trying to # guess where to put our files on a per-platform basis. ZOPE_INSTALL_SCHEME = { 'purelib': '$base/lib/python', 'platlib': '$base/lib/python', 'headers': '$base/lib/python', 'scripts': '$base/bin', 'data' : '$base/lib/python', } That looks a little evil to me. a. -- Adrian van den Dries avdd@flow.com.au Development team www.dev.flow.com.au FLOW Communications Pty. Ltd. www.flow.com.au

On Fri, 2003-06-20 at 01:38, Adrian van den Dries wrote:

You may be interested in Kenneth Almquist's ash (aka dash in Debian):

Optimally the configure script will work in any bourne-shell-derived shell (e.g. the bourne shell on Solaris).

With the distutils, ``--home`` is version-agnostic (installs package ``foo`` into ``$HOME/lib/python/foo/``) and ``--prefix`` is version-aware (installs into ``$PREFIX/lib/pythonX.Y/site-packages``).

OK. This option always seemed a little weird to me, but let's run with it. ;-)

``inst/Makefile.in`` could probably be updated to do a --prefix install if $(PREFIX) is defined, else it defaults to a --home install.

(Mockups)

# the next commands are all equivalent # and produce the install target # ``python setup.py --home=/usr/local/zope``

python inst/configure.py --home=/usr/local/zope ./configure --home=/usr/local/zope ./configure

OK, looks good.

# the next command produces the install target # ``python setup.py --prefix=/usr/local`` # which installs packages to /usr/local/lib/python2.2/site-packages

python inst/configure.py --prefix=/usr/local

I like it because it's consistent with the distutils flags. Where do ancillary files like those in bin, doc, and whatnot go when you invoke this command? /usr/local/bin and /usr/local/doc?

(Couple of hours later).

OK, attached is a patch to inst/Makefile.in and inst/configure.py that distinguishes --home and --prefix parameters and passes them onto the distutils. But for some reason, there is still no difference between --prefix and --home, even though it's passing those options to the distutils. I suspect it might have something to do with this exerpt from setup.py::

Thanks for the work! I also like the structured text style colons! ;-)

# We create a custom "install scheme" that works the same way on all # platforms. We do this in order to prevent distutils from trying to # guess where to put our files on a per-platform basis.

ZOPE_INSTALL_SCHEME = { 'purelib': '$base/lib/python', 'platlib': '$base/lib/python', 'headers': '$base/lib/python', 'scripts': '$base/bin', 'data' : '$base/lib/python', }

That looks a little evil to me.

Evil? That's a hundred dollar line of code right there! ;-) Convincing distutils to behave in a predictable way is hard. Distutils works very differently on Windows and UNIX. --home is not supported on Windows and the default filepaths are different for a prefix install between platforms. I have also experienced differences in distutils behavior between Python versions that make it difficult to know exactly what's going to happen when you run "setup.py install". I don't remember the details, but it wasn't pretty. The ZOPE_INSTALL_SCHEME was meant to gloss over the differences in OS and Python version. In this scheme, everything (data, scripts, libraries) will always be installed into a single directory regardless of your platform. If we're no longer going to punt in this way, someone (probably me) needs to do the work to ensure that we have all the options we need but that it continues to work both under UNIX and Windows. Your code is a start, but obviously we'll need to do some work on setup.py. Thanks! - C

On Fri, 2003-06-20 at 03:55, Luca - De Whiskey's - De Vitis wrote:

...

Am I missing anything?

I'd like to have the possibility to install any architecture dependant files in an different tree.

I'm afraid I'll need to understand more about what debian considers "architecture dependent". Can you provide details about what this means?

OK, thanks, I thought this was it. In Zope's case, this just implies compiled Python libraries. This can normally be specified via the --platlib flat to setup.py, but as described in a prior message in this thread, the Zope setup.py overrides platlib in order to provide X-platform compatibility for install locations. This is the thing that will need to change. Thanks! - C On Fri, 2003-06-20 at 12:19, Luca - De Whiskey's - De Vitis wrote:

On Fri, Jun 20, 2003 at 10:58:54AM -0400, Chris McDonough wrote:

...

I'm afraid I'll need to understand more about what debian considers "architecture dependent". Can you provide details about what this means?

[There is nothing special about Debian and architecture dependent files]

Dependent is any file which cannot be used on a different hardware architecture. Compiled 'c' code is architecture dependent. Python compiled source (.pyc, .pyo), text files of any sort and almost all kinds of data files like sounds, images or videos do behave the same, regardless the hardware architecture they are installed on: thus they are architecture independent.

I tryed any conbination of setup.py options to make it work, but i was not succesful: that led me to think this feature was not considered.

thanks, -- Luca - De Whiskey's - De Vitis | Elegant or ugly code as well aliases: Luca ^De [A-Z][A-Za-z\-]*[iy]'\?s$ | as fine or rude sentences have Luca, a wannabe ``Good guy''. | something in common: they local LANG="it_IT@euro" | don't depend on the language.

_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )

On June 19, PieterB wrote:

Could that make-driven software be made public? I'm currently trying to create similar Makefiles (for a new FreeBSD zope port) and would be interested in using ZC files.

This sort of stuff is almost definitely deployment-specific (and quite likely ZC proprietary property - that is much of their revenue!)

How do those files compare to the buildscript: http://cvs.zope.org/NZO_SiteLayout/buildout_zope_sandbox?cvsroot=Zope.org

Jeebers, someone sure has an aversion to shell functions! ;-) a. -- Adrian van den Dries avdd@flow.com.au Development team www.dev.flow.com.au FLOW Communications Pty. Ltd. www.flow.com.au

On Thu, 2003-06-19 at 10:14, Chris McDonough wrote:

On Thu, 2003-06-19 at 02:57, PieterB wrote:

...

Chris wrote:

...

We have make-driven software that creates us a tree via a single command by checking out various version of Python, Zope, etc. from CVS and compiling and installing them. Different versions of Zope and Python, etc. can be installed in opt and we use symlinks to manage versioning. Could that make-driven software be made public? I'm currently trying to create similar Makefiles (for a new FreeBSD zope port) and would be interested in using ZC files.

How do those files compare to the buildscript: http://cvs.zope.org/NZO_SiteLayout/buildout_zope_sandbox?cvsroot=Zope.org

The NZO make-driven buildout is an early revision of what we use. There is no publically available later revision, but it hasn't changed all that much.

Scratch that! I hadn't looked at it in a while. This looks nothing whatsoever like what our Makefile-driven buildout for NZO used to look like. I guess Sidnei didn't like it and changed it to a shell-driven buildout. The old version should be floating around somewhere... ah, there is is: http://cvs.zope.org/SiteLayout/?cvsroot=Zope.org. - C

HTH,

- C

_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )

There's only one possible way! A-A-P! (A good match for Ape, Shane ;) It's a replacement for make by Bram Moolenaar, the author of Vim, and it looks like it does a lot of things Right. http://www.a-a-p.org/index.html In fact, Aap would fit very well with Gentoo. Gentoo's emerge system takes care of package metadata, fetching the source, and unpacking it. Once it's unpacked, it gets compiled, usually with the standard .configure; make; make install steps. Individual packages could use aap instead of make. Actually, Aap also takes care of the fetching phase. Then Gentoo installs the compiled package, keeping track of what files went where, so that they can be uninstalled, and queried for what package they belong to. This Aap doesn't do, AFAIK. -- Jean Jordaan http://www.upfrontsystems.co.za

http://www.a-a-p.org/index.html This looks quite good! It has support python 2.2 support as well. I'll see if I can play with it.

Pieter

On Friday 20 June 2003 01:19 am, Jean Jordaan wrote:

...

There's only one possible way! A-A-P! (A good match for Ape, Shane ;) It's a replacement for make by Bram Moolenaar, the author of Vim, and it looks like it does a lot of things Right. Sorry, I haven't really been paying attention so this might be completely OT. It *sounds* like it's being suggested that we replace "make" (given the above statement). Has anyone used SCons? http://www.scons.org/ Richard

I think the default Zope install should not have dependencies other than that Python is required and the user has some shell system (bash/sh/MS batchfiles). On the other hand some packages (such as my zopetest, or other rpm-alike things), may require other things. I like A-A-P, because I think that would make my installerfiles much cleaner. Aap has integrated support to fail if a program returns an error, which isn't the case for shell scripts. Make also has error support and can be used to define 'modules'/'tasks' but it's terrible to create something similar to http://cvs.zope.org/NZO_SiteLayout/buildout_zope_sandbox?cvsroot=Zope.org in Makefiles. I'm trying to create a product that would build Zope sandboxes with various products (CMF, Plone, Zwiki, etc). That's a different type of requirement than ZC building a production stable Zope. I think it's a basic "seperation of concerns" issue. The Zope install should facilitate users, and packagers to install a Zope instance. Other Zope installers/pacakers/etc. can do other things as they please. On the other hand: it would be nice if there is a "standard"-layout when people would like to build zope "in-place". That would make it easier to document, and would make the life of sysadmin's running Zope on more than one platform much easier. About Scons: I never heard of it before, but it's not suitable for my task. I want to create something that can easily interact with FreeBSD ports, and what has Python2 support, and is more stable than 0.14 alpha which is written in Python 1.5.2

...

I think the default Zope install should not have dependencies other than that Python is required and the user has some shell system (bash/sh/MS batchfiles).

... and aap apparently ;)

I'm thinking that ZC needs a more capable make replacement. That isn't quite the topic of this thread, but Aap (eg.) would be cool already if it was just used by ZC to manage their checkout/builds, and was available as option for outsiders to use. The releases needn't have a new dependency.

It's based on the winning design for software construction tools (ie. "make" replacement) in the Software Carpentry contest

Yes, like Roundup :)

It's certainly been around for longer than aap :)

I'd bet that Bram has been brooding on Aap for a long time, and he certainly has masses of experience managing complex build environments, with Vim compiling on about a million architectures with or without myriads of compilation options, language bindings, GUI toolkits, etc.

...

, and is more stable than 0.14 alpha

I note with a grin that up until this month aap was at release 0.150 :)

That's the hundred and fiftieth release, ne :) -- Jean Jordaan (using WindowMaker 0.80.2) http://www.upfrontsystems.co.za

It *sounds* like it's being suggested that we replace "make"

That's correct, though Aap can usefully do much more than make, such as fetching remote sources and managing CVS checkouts/-ins.

Has anyone used SCons? http://www.scons.org/

Well, they feature neck-and-neck in July, so if someone (or someone someone knows) is around there, they could compare and ask pointed questions: 2003 May 15: BOF session at O'Reilly conference The proposal to organize a BOF session on A-A-P has been accepted. It will take place on Thursday evening July 10, from 8 to 9 pm. This is directly after the BOF session on SCons, in the same room. A nice occasion for people to talk about modern building tools! More information about the conference here. (links at http://www.a-a-p.org/index.html ) -- Jean Jordaan http://www.upfrontsystems.co.za

On Thursday 19 June 2003 15:58, Shane Hathaway wrote:

Has anyone tried extracting Gentoo's build system and using it for partial software distributions? That might be the way to go.

I am now using Gentoo on all my servers and on this workstation. I find value in being able to manage Zope stuff as a non-root user. Portage was designed to allow this, but I doubt it is well exercised. Trying this has been on my to-do list for a while. -- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson

--On Thursday, June 19, 2003 10:07:52 +1000 Adrian van den Dries <avdd@flow.com.au> wrote:

...

I'd advise against installing Zope library files into site-packages unless you put them in a site-packages subdirectory (like site-packages/zope270 or site-packages/zope271). Otherwise there will be no way to run multiple Zope versions on the same machine sanely because different Zope versions have different libraries.

May I respectfully ask why there is so much concern with such complicated setups? Surely a production environment (which is what any Zope distribution should aim for) will standardise on a software version?

One word: migration As I've found out the hard way, there are times when it is much easier to conduct server upgrades if you can have different versions of Zope installed on a server at the same time. This is especially true in budget constrained colocation environments where adding additional machine for the upgrade would cost money. Dan Pierson

On Thu, 2003-06-19 at 09:53, Luca - De Whiskey's - De Vitis wrote:

On Wed, Jun 18, 2003 at 03:08:12PM -0400, Chris McDonough wrote: I'm both disappointed and happy for the same reason :) When 2.7 will be out i'll adapt my zopectl to the new Zope installation procedure.

Great.

I'll keep it a separate package, and i surely take advantage of your help.

Excellent. Thanks again, - C

On Wed, 2003-06-18 at 19:26, Adrian van den Dries wrote:

I think most of us would agree that .py(c) files are *libraries* and not *data files*. Data files would be the skeleton instance directory.

To make matters clear, my concern about adhering to the FHS is limited to offering features which make it easy for people to build binary distributions which more or less allow you to adhere to the FHS from an invokation of "make install" from the source tarball. The result of a default invocation of "./configure; make; make install" of Zope will almost certainly never adhere to the FHS, but if I can make it easier by providing file install location options to "configure" that make it possible to install an FHS-compliant Zope "software home", I'll do so. Offering these options would make it possible for the binary-building software to not do too much work itself copying files around and whatnot; instead it could just invoke configure, make, make install with certain option flags set during configure. Instance creation is a different matter; we can deal with that separately if the current 'mkzopeinstance' doesn't have the necessary options. I just need to know what the requirements are and I'll do my best to service them. - C

On June 18, Chris McDonough wrote:

And Guido, for zdctl.

Will Zope use zdctl? It would be cool to have the same framework used to control individual instances (Zope or ZEO), and zopectl can become an external zdaemon instance manager (I'll call it z*ctl). Zope and ZEO simply provide start hooks for zdaemon. So, you configure a ZEO instance and a Zope instance, add those instances to a global configuration file, and ask z*ctl to start the instances (if you have z*ctl) or you start them individually with zdctl.py. Hrm, applications that are really plugins to a generalised daemon application. That sounds familiar. ;-) a. -- Adrian van den Dries avdd@flow.com.au Development team www.dev.flow.com.au FLOW Communications Pty. Ltd. www.flow.com.au

2. I see no reason for Zope to eschew the .py extension for callable scripts, eg. configure.py, zopectl.py, etc. Distributions may choose to wrap these in shell scripts, or strip the extension, but that is a distribution's decision. $(insert impertinent aside about Zope's support for Python Labs and branding.)

I'm not sure what you mean by that final remark, but I also like to install scripts with their .py extension. Some at PL disagree though. --Guido van Rossum (home page: http://www.python.org/~guido/)