[Zope-CMF] GenericSetup proposal: upgrade profiles

[Maurits van Rees]

Hi,

Products.GenericSetup/interfaces.py says:
BASE, EXTENSION = 1, 2

I propose adding another one:
UPGRADE = 3
and possibly a fourth one:
UNINSTALL = 4


Intro
-----

This refs a small discussion on the Plone developers list at the end
of May, which can be interesting (but not necessary) as background:
http://plone.org/support/forums/core#nabble-tc2966873|a2973467

The meaning should be reasonably clear from the names:

- BASE: completely new install; purge existing settings; usually
  applied once during the lifetime of the website: at creation time

- EXTENSION: extend a BASE profile; do not purge existing settings;
  might get applied multiple times during the lifetime of the website.

- UPGRADE: upgrade for a BASE or EXTENSION profile; no purge; usually
  applied once.  Actually, it does not need to be tied to another
  profile in any way: it is just a bunch of import steps that need to
  be applied once.

- UNINSTALL: undo what was done while applying another profile (BASE,
  EXTENSION or less likely an UPGRADE); no purge; usually applied once:
  when uninstalling the product that the profile belongs to.

- Hm, let's throw TEST (value 5) in as an idea as well: a non-purging
  profile that is meant to be applied during the automated tests but
  not during normal use.  Can anyone come up with a use case?

- And I suddenly wonder if it is handy to define SNAPSHOT (value 6)
  here too.  Might this make some of the special handling for
  snapshots easier or unnecessary?  If someone wants to plea for that:
  go ahead. :)

I do not have plans for UNINSTALL, TEST or SNAPSHOT, but I think
UPGRADE would be handy, so let's concentrate on that one.


Proposal
--------

- Add UPGRADE to the GS interfaces.py, for use in the 'provides' line
  of a profile registration in zcml:

    provides="Products.GenericSetup.interfaces.UPGRADE"

- Show only BASE and EXTENSION profiles and not UPGRADE (or other)
  profiles in some parts of the UI, so that those parts are not
  cluttered by too many profiles.

Third party products can then register one true EXTENSION profile that
is meant to be applied when installing the product, and several
UPGRADE profiles meant to be applied once, without confusing for
example the CMFQuickInstaller in Plone.


Use case
--------

For our use case we assume that you have made an add-on product.  It
has an EXTENSION profile.  You have applied the profile in a website.
Now you want to add a catalog index.  Your options are:

A. Add this in the current extension profile.  This means that any
   time you apply this profile this index will be removed and
   recreated so you lose data; you need to reindex the index (manually
   or in python code - not GenericSetup) and that takes time.

B. Add an upgrade step.  Works fine, but this needs python code along
   these lines:

     indexes = catalog.indexes()
     if 'new' not in indexes:
         catalog.addIndex('new', 'some_meta_type')
         catalog.manage_reindexIndex(ids=['new'])

   But why add python code when there is already a perfectly fine
   import handler that can read a catalog.xml?  We are back to the
   dark days before there was GenericSetup then.  (Well, okay, writing
   python code is not *that* bad. ;-))

C. Add it to the current extension profile.  Do not apply this profile
   ever again: only once at install time.  Instead add an upgrade step
   that applies the catalog step from this profile.  But this has
   basically the same problem as option A: if you later add a second
   catalog index you would need to create a new upgrade step to apply
   the catalog step and then your first index would be removed and
   recreated empty again.

D. Define a new EXTENSION profile that just has a catalog.xml.  Either
   go to the import tab of portal_setup to run all steps of that
   profile, or create an upgrade step that does this.  But this means
   that in several spots the list of available profiles gets longer,
   which threatens to make it hard to use.

   The migration machinery of CMFPlone is moving to the
   plone.app.upgrade package (not released yet), which already defines
   about 24 upgrade steps with 11 extension profiles and that will
   only increase, potentially one for each point release of Plone.
   See http://svn.plone.org/svn/plone/plone.app.upgrade/trunk

   (And the CMFQuickInstaller can get confused when a package has more
   than one profile; but that is not a problem for this mailing list,
   and I expect that in Plone 4 either the CMFQuickInstaller is gone
   or it ends up as just a small wrapper around GenericSetup.)

Similar arguments could be made about adding a property to a property
sheet: you only want to do that once and not override any changes made
to that property by a site admin when you reapply the profile.

So I think another option would help:

E. Define an UPGRADE profile that just has a catalog.xml.  Going to
   the import tab of portal_setup should not list this profile, but
   only EXTENSION and BASE profiles, so that list is not so cluttered
   anymore.  You now just create an upgrade step that applies the
   profile.


Implementation
--------------

The biggest changes (which are not actually that big) in GenericSetup
code would be:

- in tool.py make sure that purge is only True for BASE profiles:

    def _getImportContext(...):
        ...
        #should_purge = (info.get('type') != EXTENSION)
        should_purge = (info.get('type') == BASE)

    def listContextInfos(self):

- In registry.py (and tool.py) allow listing only some types, for
  example here:

    def listProfiles( self, for_=None, types=None ):
        if types is None:
            # possibly add SNAPSHOT here.
            types = [BASE, EXTENSION]
        result = []
        for profile_id in self._profile_ids:
            info = self.getProfileInfo( profile_id )
            if for_ is None or issubclass( for_, info['for'] ):
                if info['type'] in types:
                    result.append( profile_id )
        return tuple( result )

And then some template changes and of course tests.

As a bonus it may be handy to combine the zcml registration of the
upgrade step and the upgrade profile, but that will be technically
more difficult.  Something like this:

  <genericsetup:upgradeStep
      title="Add catalog index"
      description=""
      source="1"
      destination="2"
      directory="profiles/upgrade2"
      name="upgrade2"
      />

This would then register directory profiles/upgrade2 as an UPGRADE
profile with name 'upgrade2' and register an upgrade step that applies
that profile.  But that is icing on the cake and I am not sure if we
need more zcml constructs here.


I could do these changes (on a branch of course).


Questions
---------

Is this feature interesting for inclusion in GenericSetup?

Is it in scope for 1.5.x?

Is anyone already registering profiles using numbers other than 1
(BASE) or 2 (EXTENSION)?

Do you see a use case for UNINSTALL, TEST, or SNAPSHOT?

Other comments?


Cheers,

-- 
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]