Hi all,
zope.catalog is a crucial part of ZTK, as it adds the equivalent of RDBMS SELECT queries to our ODB based apps. However in my opinion its default setup is suboptimal as a generic library in ZTK, because it is hardwired up for ZMI "point and click" operation. As soon as you create and register an ICatalog utility, every ObjectAddedEvent and ObjectModifiedEvent will trigger stuffing the object into intids, and each intid registration event will trigger an attempt to index an object. When you add an index to the catalog, it gets reindexed automatically. Because these behaviours are implemented by event handlers, there is no way for an app to override or disable them.
There is an "opt out" mechanism for keeping objects out of the catalog (INoAutoIndex, INoAutoReindex marker interfaces), but curiously the latter does not inherit from the former,so your object can be indexed on modification even if you made it provide INoAutoIndex.
Automatically stuffing all newly added objects into the int ids utility and then further into the catalog is bad, because, first, it cripples any alternative uses of the int ids utility, turns it into purely a catalog accessory, and second, places some restrictions on containers and contained objects due to the persistent oid based key reference implementation.
About a year ago zope.app.catalog as been moved to zope.catalog. I think during this move there was a unique opportunity to leave these event handlers behind in zope.app.catalog, so that the "no .app" version is free of these forced choices. But perhaps it's not too late to do in now? After all, no stable version of Zope 3 / ZTK codebase has been released after this move? People relying on automatic indexing would have to register a couple of event handlers in their configurations (or include zope.app.catalog), but the rest of us would be able to choose the policy of how and when the objects are registered with IntIds and indexed in the catalog.
Albertas
We teased out all this ZMI- and event-handling related behavior in repoze.catalog. r.catalog consumes no events, and disuses the IIntIds utility. It's otherwise a lot like zope.catalog. It might be a good starting point if you want more control.
FWIW, we use this code heavily in several production apps, it's not a "throw it over the wall" sort of things. It has good test coverage.
http://pypi.python.org/pypi/repoze.catalog/0.7.1
Albertas Agejevas wrote:
Hi all,
zope.catalog is a crucial part of ZTK, as it adds the equivalent of RDBMS SELECT queries to our ODB based apps. However in my opinion its default setup is suboptimal as a generic library in ZTK, because it is hardwired up for ZMI "point and click" operation. As soon as you create and register an ICatalog utility, every ObjectAddedEvent and ObjectModifiedEvent will trigger stuffing the object into intids, and each intid registration event will trigger an attempt to index an object. When you add an index to the catalog, it gets reindexed automatically. Because these behaviours are implemented by event handlers, there is no way for an app to override or disable them.
There is an "opt out" mechanism for keeping objects out of the catalog (INoAutoIndex, INoAutoReindex marker interfaces), but curiously the latter does not inherit from the former,so your object can be indexed on modification even if you made it provide INoAutoIndex.
Automatically stuffing all newly added objects into the int ids utility and then further into the catalog is bad, because, first, it cripples any alternative uses of the int ids utility, turns it into purely a catalog accessory, and second, places some restrictions on containers and contained objects due to the persistent oid based key reference implementation.
About a year ago zope.app.catalog as been moved to zope.catalog. I think during this move there was a unique opportunity to leave these event handlers behind in zope.app.catalog, so that the "no .app" version is free of these forced choices. But perhaps it's not too late to do in now? After all, no stable version of Zope 3 / ZTK codebase has been released after this move? People relying on automatic indexing would have to register a couple of event handlers in their configurations (or include zope.app.catalog), but the rest of us would be able to choose the policy of how and when the objects are registered with IntIds and indexed in the catalog.
Albertas
Hi there,
Albertas Agejevas wrote: [snip]
About a year ago zope.app.catalog as been moved to zope.catalog. I think during this move there was a unique opportunity to leave these event handlers behind in zope.app.catalog, so that the "no .app" version is free of these forced choices.
Doing this would have needed a migration strategy more complicated than "Oh, I'm just going to lose the ZMI when I switch" and people's applications somehow would have to gain registrations manually. I guess a damn good changelog would've helped somewhat..
In general, I think it's not too bad we lose the ZMI in a step separately from other policies. Changing too much in one step could be more disruption than the community can manage.
But perhaps it's not too late to do in now? After all, no stable version of Zope 3 / ZTK codebase has been released after this move? People relying on automatic indexing would have to register a couple of event handlers in their configurations (or include zope.app.catalog), but the rest of us would be able to choose the policy of how and when the objects are registered with IntIds and indexed in the catalog.
Would you propose the same treatment for zope.intid as well?
Would this involve removing the configure.zcml in these packages, or what is your proposal exactly?
Again, it needs damn good documentation if we were to go ahead with this, including details on how to register event handlers to accomplish the previous behavior. Both the changelog and the documentation that appears on pypi would need to be updated with instructions about what is going on. This change would also definitely be worth a feature release.
+1 if documentation is good.
Regards,
Martijn
On Tue, Sep 29, 2009 at 01:09:08PM +0200, Martijn Faassen wrote:
About a year ago zope.app.catalog as been moved to zope.catalog. I think during this move there was a unique opportunity to leave these event handlers behind in zope.app.catalog, so that the "no .app" version is free of these forced choices.
Doing this would have needed a migration strategy more complicated than "Oh, I'm just going to lose the ZMI when I switch" and people's applications somehow would have to gain registrations manually. I guess a damn good changelog would've helped somewhat..
Correct me if I'm wrong, but zope.app.catalog still remains with the ZMI bits and re-imports of the zope.catalog functionality. So apps depending on zope.app.catalog are not affected at all. I'm saying that some of the event handlers (or their ZCML registrations) could have stayed there as well.
But perhaps it's not too late to do in now? After all, no stable version of Zope 3 / ZTK codebase has been released after this move? People relying on automatic indexing would have to register a couple of event handlers in their configurations (or include zope.app.catalog), but the rest of us would be able to choose the policy of how and when the objects are registered with IntIds and indexed in the catalog.
Would you propose the same treatment for zope.intid as well?
I don't know, the subscribers in zope.intid don't bother me too much.
Would this involve removing the configure.zcml in these packages, or what is your proposal exactly?
In essence, I would like to see zope/catalog/subscribers.zcml removed from the default zope/catalog/configure.zcml.
Again, it needs damn good documentation if we were to go ahead with this, including details on how to register event handlers to accomplish the previous behavior. Both the changelog and the documentation that appears on pypi would need to be updated with instructions about what is going on. This change would also definitely be worth a feature release.
+1 if documentation is good.
:D
Albertas
Albertas Agejevas wrote:
On Tue, Sep 29, 2009 at 01:09:08PM +0200, Martijn Faassen wrote:
About a year ago zope.app.catalog as been moved to zope.catalog. I think during this move there was a unique opportunity to leave these event handlers behind in zope.app.catalog, so that the "no .app" version is free of these forced choices.
Doing this would have needed a migration strategy more complicated than "Oh, I'm just going to lose the ZMI when I switch" and people's applications somehow would have to gain registrations manually. I guess a damn good changelog would've helped somewhat..
Correct me if I'm wrong, but zope.app.catalog still remains with the ZMI bits and re-imports of the zope.catalog functionality. So apps depending on zope.app.catalog are not affected at all. I'm saying that some of the event handlers (or their ZCML registrations) could have stayed there as well.
We hope to deprecate zope.app.catalog entirely eventually, so we should support people as much as possible in moving over to zope.catalog. Just removing the ZMI stuff is a relatively straightforward deprecation step. Removing configuration essential to the functioning of many applications is less so, which is why I'm saying it's not bad we do this in two steps.
Would this involve removing the configure.zcml in these packages, or what is your proposal exactly?
In essence, I would like to see zope/catalog/subscribers.zcml removed from the default zope/catalog/configure.zcml.
Ah, yeah, that would make documenting things easier too: just include zope/catalog/subscribers.zcml if you them.
Regards,
Martijn
On Sun, Sep 27, 2009 at 1:16 PM, Albertas Agejevas alga@pov.lt wrote:
About a year ago zope.app.catalog as been moved to zope.catalog. I think during this move there was a unique opportunity to leave these event handlers behind in zope.app.catalog, so that the "no .app" version is free of these forced choices. But perhaps it's not too late to do in now? After all, no stable version of Zope 3 / ZTK codebase has been released after this move? People relying on automatic indexing would have to register a couple of event handlers in their configurations (or include zope.app.catalog), but the rest of us would be able to choose the policy of how and when the objects are registered with IntIds and indexed in the catalog.
Is there any reason you don't just ignore the configure.zcml in that package?
Nobody forces you to load the zcml file. The whole point of having these policies in zcml is to be able to ignore or override them easily.
Hanno
On Tue, Sep 29, 2009 at 01:33:21PM +0200, Hanno Schlichting wrote:
Is there any reason you don't just ignore the configure.zcml in that package?
Nobody forces you to load the zcml file. The whole point of having these policies in zcml is to be able to ignore or override them easily.
Good point, thanks! I haven't looked at the configure.zcml for zope.catalog previously. In fact the config for zope.catalog is split into several files, so it's very easy to leave out the subscribers but include the rest.
Albertas
Albertas Agejevas wrote:
People relying on automatic indexing would have to register a couple of event handlers in their configurations (or include zope.app.catalog), but the rest of us would be able to choose the policy of how and when the objects are registered with IntIds and indexed in the catalog.
+1 from me: reusable library code shouldn't be imposing policy on the application.
Tres.
-
Albertas Agejevas -
Chris McDonough -
Hanno Schlichting -
Martijn Faassen -
Tres Seaver