[Zope-CMF] GenericSetup proposal: upgrade profiles

[Rob Miller]

i'm not actively working on GenericSetup these days, so i don't know that my 
opinion counts for much, but i'm -1 on this.  to me it seems to introduce an 
entirely new way of handling upgrades that is likely to confuse people. 
GenericSetup is more than complex enough already, IMO.

Maurits van Rees wrote:
> Use case
> --------
> 
> For our use case we assume that you have made an add-on product.  It
> has an EXTENSION profile.  You have applied the profile in a website.
> Now you want to add a catalog index.  Your options are:
> 
> A. Add this in the current extension profile.  This means that any
>    time you apply this profile this index will be removed and
>    recreated so you lose data; you need to reindex the index (manually
>    or in python code - not GenericSetup) and that takes time.

in my mind this is the core problem, that there is no way to non-destructively 
apply an index to a catalog.  this is a weakness in GenericSetup, but 
developing an entirely new alternative mechanism for handling upgrades seems 
to me the wrong solution.

> B. Add an upgrade step.  Works fine, but this needs python code along
>    these lines:
> 
>      indexes = catalog.indexes()
>      if 'new' not in indexes:
>          catalog.addIndex('new', 'some_meta_type')
>          catalog.manage_reindexIndex(ids=['new'])
> 
>    But why add python code when there is already a perfectly fine
>    import handler that can read a catalog.xml?  We are back to the
>    dark days before there was GenericSetup then.  (Well, okay, writing
>    python code is not *that* bad. ;-))
> 
> C. Add it to the current extension profile.  Do not apply this profile
>    ever again: only once at install time.

this confuses me.  in my mind, extension profiles should be idempotent.  IOW, 
it should _always_ be safe to re-apply an extension profile to a site, 
assuming that extension profile does indeed represent the final state that 
you'd like the site to have.

>  Instead add an upgrade step
>    that applies the catalog step from this profile.  But this has
>    basically the same problem as option A: if you later add a second
>    catalog index you would need to create a new upgrade step to apply
>    the catalog step and then your first index would be removed and
>    recreated empty again.

again, this seems to me a weakness of the catalog index importing, and it 
should be fixed there.

it is option (C) which i would recommend, so much so that there is even a 
dedicated ZCML tag specific for this purpose: genericsetup:upgradeDepends. 
here's an example from the GenericSetup tests:

       ...       <genericsetup:upgradeDepends
       ...           title="Bar Upgrade dependency profile import steps"
       ...           description="Re-imports steps from the profile"
       ...           import_profile="profile-Products.CMFDefault:default"
       ...           import_steps="baz bat"
       ...           run_deps="True"
       ...           purge="True"
       ...           />

upgradeDepends lets you specify any arbitrary import steps from any arbitrary 
profile, so that you don't have to repeat the boilerplate code of invoking 
those import steps by hand whenever you need to write such an upgrade step.

> D. Define a new EXTENSION profile that just has a catalog.xml.  Either
>    go to the import tab of portal_setup to run all steps of that
>    profile, or create an upgrade step that does this.  But this means
>    that in several spots the list of available profiles gets longer,
>    which threatens to make it hard to use.

i think the use of profiles for upgrades was an understandable stop-gap when 
GS did not yet provide any other mechanism for handling ugprades.  doing so 
now is, IMO, a mistake, even if you designate a specific type of profile for 
this purpose.

>    The migration machinery of CMFPlone is moving to the
>    plone.app.upgrade package (not released yet), which already defines
>    about 24 upgrade steps with 11 extension profiles and that will
>    only increase, potentially one for each point release of Plone.
>    See http://svn.plone.org/svn/plone/plone.app.upgrade/trunk
> 
>    (And the CMFQuickInstaller can get confused when a package has more
>    than one profile; but that is not a problem for this mailing list,
>    and I expect that in Plone 4 either the CMFQuickInstaller is gone
>    or it ends up as just a small wrapper around GenericSetup.)
> 
> Similar arguments could be made about adding a property to a property
> sheet: you only want to do that once and not override any changes made
> to that property by a site admin when you reapply the profile.
> 
> So I think another option would help:
> 
> E. Define an UPGRADE profile that just has a catalog.xml.  Going to
>    the import tab of portal_setup should not list this profile, but
>    only EXTENSION and BASE profiles, so that list is not so cluttered
>    anymore.  You now just create an upgrade step that applies the
>    profile.

so if your product requires this new index, so much so that you made an 
upgrade profile for it, it's reasonable to assume that you'd want any new 
installations of this product to also have this new index.  does this mean 
that when you apply an extension profile, all upgrade profiles are 
automatically applied?  or do you have to add the index to the extension 
profile as well?

maybe this horse is already dead, but i think you're trying to solve the wrong 
problem.  it's a PITA that you have to reindex after importing a catalog index 
into a catalog.  i think the catalog import logic should be improved, so that 
a) the data in the index isn't wiped out unless it has to be b/c the index has 
actually changed and b) if any reindexing does need to happen, there is an 
option to have this happen automatically after the rest of the profile has 
been applied.

-r