[Zope-CMF] Re: [dev] RFC: rethinking GenericSetup extension profiles

[Martin Aspeli]

yuppie wrote:

> Since CMF 1.5.1 CMFSetup/GenericSetup supports extension profiles. They 
> are quite useful, but their current format has some fundamental issues:

I posted something in a similar vein to this on plone-developers last 
night, about using GenericSetup profiles for product installation and 
un-installation. The basic premise is that people install Plone (which 
has a base profile) and the install and un-install products, trying them 
out. They do this on sites they intend to keep (even if they're just 
test instance) and may not want to re-create the site from scratch 
(including content!) each time they decide they don't like some simple 
product.

See 
http://www.nabble.com/Installing-products-using-GenericSetup-tf1995512.html


> 1.) Extension profiles are hard to create and maintain

+1 - I've just been learning to do this and intend to document it, but 
there are some conceptual barriers if nothing else. The thing that gets 
to me is that when you want to apply an extension profile (i.e. get the 
extra functionality it installs) you click "run all import steps" after 
applying the profile. It seems very counter-intuitive (and potentially 
dangerous) to re-run e.g. all workflow installation for all products 
when I only wanted the new workflows provided by my new product that 
registered that new extension profile.

> You need a lot of knowledge about the import handlers. Only if you know 
> how their non-purging mode is implemented and which update directives 
> are available you can write good extension profiles. There are no tools 
> that help you to create extension profiles.

Bearing in mind I don't understand purging and non-purging modes, take 
that as a yes :)

> 2.) Sites using extension profiles are hard to create and maintain
> 
> You have to know which profiles depend on other profiles.

This was another problem I hit. It seemed to me profiles were being 
applied alphabetically by the Plone ZMI setup form, but who knows ;-)

> It's hard to 
> control the order in which profiles are applied. The Comparison tab 
> becomes useless if the site uses several extensions that are not part of 
> the 'initial_configuration' snapshot. (Sometimes I create a new site to 
> get a clean snapshot of the applied profiles and delete that site again. 
> A quite expensive procedure.) A snapshot contains no information which 
> original profiles it 'contains'. The tool has no control on the applied 
> profiles. There is no way to uninstall profiles.

This is the other big one - we can't reasonably tell people to make 
extension profiles for their add-on products if users won't be able to 
remove them again cleanly. The irony is that the information contained 
in the GS XML files should make it way easier to figure out how to do 
clean un-installs compared to the Extensions.Install.uninstall() method 
idiom.

> And now forget everything you know about extension profiles. Imagine 
> they were like this:
> 
> Extension profiles describe how to transform base profiles. They know 
> which profiles they can transform (AKA dependencies). They are applied 
> to an other profile, *not* to the site.

That sounds sensible.

> Only complete profiles (base profiles or extended base profiles) are 
> applied to the site. The handlers only need to understand complete 
> configuration files.

That sounds easier, but also a bit scary. If I want to just install some 
product after a site has been populated with content, applying a 
complete profile implies that it may stomp on changes I've made manually 
afterwards (people click around the ZMI after reading some how-to all 
the time). And what about being able to re-install (i.e. reset to 
original state as defined by the extension profile) and un-install 
(remove/reset the state that existed before the profile) a single 
product only?

> The setup tool knows how to merge profiles without modifying the site. 
> You can compare your snapshot with any valid combination of profiles you 
> like. There might even be a function that searches for the profile 
> combination that has the smallest diff to your snapshot and finds this 
> way the best dependencies list.
> 
> 
> And now imagine we have a tool that can *create* extension profiles 
> based on the current customizations.
> 
> This would allow to provide many new features:
> 
> - customization snapshots:
> Snapshots would just contain the customizations and a list of 
> dependencies, not a complete base profile.
> 
> - install/uninstall:
> Basically any setup change is applied by recreating the site from 
> scratch. At least for some expensive to create items like indexes we 
> have to make sure we don't delete/recreate them if they were not 
> modified. Uninstall would be no different procedure, we just don't 
> reapply a specific profile.

This is the bit that worries me. Recreating the site from scratch sounds 
like it'd be a nightmare. What if some bug causes portal_setup to 
effectively wipe people's sites or people's settings? What about the 
cases where the XML vocabulary known to the setup handlers don't cover 
all possible things people do in the ZMI (as indeed they don't do now). 
I still see a real-world need for 'importVarious' type handlers that do 
installation the old-fashioned way with Python.

It also sounds quite expensive for complex sites, even with basic 
optimisations.

> - migrations:
> There are different kinds of migrations. Updates in the base profiles 
> would be applied using the install procedure.

Again, that sounds sensible for site-wide migrations. It sounds a bit 
less easy in cases where you need contingencies ("add this property if 
it didn't exist during migration, but if the user added it manually, 
don't stomp on their value") or when you want to migrate only the 
changes implied by a particular extension profile ("leave the 
customisations to the base profile alone, but apply the changes that 
have since been made to extension profile X because product X was upgraded).

Martin