[Zope-CMF] Re: GenericSetup: catalog.xml ideas

[yuppie]

Hi Maurits!


Maurits van Rees wrote:
> So here are some points, with suggested changes to the _initIndexes
> function in exportimport.py.  The tests even run after this, so it
> seems these changes do not break anything.  But I have not *added*
> tests for this.

To get your patches accepted it would be useful to have new tests that
fail without the patches.

> 1. Removing an index
> --------------------
> 
> Currently, you can add an attribute 'remove' to an index in
> catalog.xml.  This means the index will be removed.  Presumably this
> is meant to remove an index that was previously used by a product but
> is now not necessary anymore or even harmful.
> 
> When you apply this profile a second time, you run into an error.
> Here as a test I added remove="True" to the getAssignees index:
> 
>     ...
>     Module Products.GenericSetup.ZCatalog.exportimport, line 101, in _initIndexes
>     Module Products.ZCatalog.ZCatalog, line 1018, in delIndex
>     Module Products.ZCatalog.Catalog, line 262, in delIndex
>   CatalogError: The index getAssignees does not exist
> 
> And of course it is absolutely fine that this index does not exist, as
> I want it removed anyway. :)
> 
> This can be fixed with the following change to exportimport.py (around
> line 94):
> 
>              if child.hasAttribute('remove'):
> -                zcatalog.delIndex(idx_id)
> +                # Remove index if it is there; then continue to the
> +                # next index.
> +                if idx_id in zcatalog.indexes():
> +                    zcatalog.delIndex(idx_id)
>                  continue

+1

> 2. Changing an index
> --------------------
> 
> I had a FieldIndex getAssignees, which is in any Data.fs that uses the
> eXtremeManagement product.  I found out that it was better to make
> this a KeywordIndex.  So I added this to my catalog.xml:
> 
>   <index name="getAssignees" meta_type="KeywordIndex">
>    <indexed_attr value="getAssignees"/>
>   </index>
> 
> The only effect this has is that the index is cleared: all values are
> gone (which will be my third point below).  The index itself is still
> a FieldIndex, not the KeywordIndex that I requested.
> 
> So I propose a new attribute that can be used in catalog.xml:
> 'purge'.  When this attribute is set, the index gets removed and then
> readded, which will mean that in my case the index would become a
> KeywordIndex like I wanted.
> 
> This can be done with the following change (around line 100):
> 
> +            if child.hasAttribute('purge'):
> +                # If the index is there, remove it; this can be used
> +                # to remove an index with the same name but a wrong
> +                # meta_type.
> +                if idx_id in zcatalog.indexes():
> +                    zcatalog.delIndex(idx_id)
> +
>              if idx_id not in zcatalog.indexes():
> 
> 
> This is not ideal, as it always destroys the index, even when it is
> already correctly there, but it is an improvement.

-1

This requires an extra line, but the result should be the same:

    <index name="getAssignees" remove="True">
    <index name="getAssignees" meta_type="KeywordIndex">
     <indexed_attr value="getAssignees"/>
    </index>

> 3. Keeping an index
> -------------------
> 
> When you mention an index in your catalog.xml, GenericSetup clears it:
> after applying the profile, the index has zero values.  For me it
> seems more logical to keep the index if it is already there.  In the
> current situation, I have learned to work around this by adding a
> dependency step with some code that reindexes those cleared indexes.
> 
> This can take a long time so I would like to avoid doing that on each
> reinstall, especially when I am busy developing this product and am
> often reinstalling it or reapplying the profile.
> 
> So I propose a new attribute that can be used in catalog.xml:
> 'keep'.  When this attribute is set and the index is already there,
> just do nothing and get on with the next index.  If the index is not
> there, add it.
> 
> This can be done with the following change (around line 108):
> 
>                  meta_type = str(child.getAttribute('meta_type'))
>                  zcatalog.addIndex(idx_id, meta_type, extra)
> +            elif child.hasAttribute('keep'):
> +                continue
> 
> 
> Or maybe some attribute can be added like 'reindex' or
> 'reindex_when_new', which would trigger a reindex of the index,
> possibly only when the index was added this time and not when it was
> already there.  This might be an attribute of the catalog tool object
> itself.  This would at least avoid the need of having to add
> non-GenericSetup code to force a reindex.

-1

I prefer the next solution.

> 4. Keep when no change; purge otherwise
> ---------------------------------------
> 
> As a probably better alternative, it would be nice to do all this
> automatically without any extra attributes.
> 
> - When the settings for the index are the same in catalog.xml and in
>   the current catalog: do nothing.

+1

> - When at least one setting is different, remove the index and add it
>   with the new settings.

-0.5

Don't know how well this works right now with indexes, but it should be 
possible to override specific settings and keep the rest. Removing and 
re-adding indexes clears all settings.

Your solution has other benefits, but I'm not sure they are more 
important. And changing that behavior might break some existing profiles.

> This seems more difficult to implement though.
> 
> 
> Does this make sense?
> 
> Or has this been debated to death already without me knowing it? :)

Well. There is an open collector issue you should know:
http://www.zope.org/Collectors/CMF/455


Cheers,

	Yuppie