[Warning] Zope 3 component architecture (CA) not reliably usable for registrations from Python
I tried to use Zope3 events to get informed when requests start and end. One of our modules (the interface module to "jpype") requires such a notification for reliable work. Therefore, it tried to register the corresponding subscriptions on import of this module. Unfortunately, this works very unreliably -- for the following reasons: The CA performs by itself only extremely minimal initialization. Especially, it does not register the "Adapters" service, necessary for the registration of event subscriptions. The full CA initialization is only performed quite late, in the call to "Products.Five.initialize". Event notification registration attempts before this time will fail (with a "ComponentLookupError: 'Adapters'"), those performed after this time will succeed. The "Five" initialization is stupid enough that it will fail when the service registration for 'Adapters' has already been performed. At the current state, event notifications can reliably only be defined via ZCML. This is unfortunate for our use case where the registration should be bound to the import of a given module. -- Dieter
Dieter Maurer wrote:
I tried to use Zope3 events to get informed when requests start and end.
One of our modules (the interface module to "jpype") requires such a notification for reliable work. Therefore, it tried to register the corresponding subscriptions on import of this module. Unfortunately, this works very unreliably -- for the following reasons:
The CA performs by itself only extremely minimal initialization. Especially, it does not register the "Adapters" service, necessary for the registration of event subscriptions.
The full CA initialization is only performed quite late, in the call to "Products.Five.initialize". Event notification registration attempts before this time will fail (with a "ComponentLookupError: 'Adapters'"), those performed after this time will succeed.
The "Five" initialization is stupid enough that it will fail when the service registration for 'Adapters' has already been performed.
At the current state, event notifications can reliably only be defined via ZCML. This is unfortunate for our use case where the registration should be bound to the import of a given module.
I'm a bit surprised why the huge [warning] is necessary. What I understand you're saying is that you cannot use the component architecture before it's done initializing. You also assert that you need a registration to happen at Python import time. I'm curious to hear more about why you need to do this - after all, you're talking about listening to request start and end, and this happens after import time. With Grok we have the grok.subscribe decorator which allows us to do this: @grok.subscribe(IFoo, IObjectModifiedEvent): def handle(obj, event): ... do something ... The decorator gets executed during import time, but it doesn't actually try registering anything into the CA. Instead it leaves an annotation on the module and these annotations are read later (at "grok time", when everything has been initialized) to make registrations in the component architecture. Perhaps you can use a similar strategy. You could even try taking a look at how we do this in Grok. Regards, Martijn
Martijn Faassen wrote at 2007-1-10 11:45 +0100:
.... I'm a bit surprised why the huge [warning] is necessary. What I understand you're saying is that you cannot use the component architecture before it's done initializing.
The warning comes from bad experience. I have tested my registration and in my test it worked because "Products.Five.initialize" happened to be called before. But during real use, my module was imported before "Products.Five.initialize" has been called -- leading to strange and not immediately recognizable errors.
You also assert that you need a registration to happen at Python import time. I'm curious to hear more about why you need to do this
I know that the module needs to get informed about request starts. What is more natural than registering for this event when the module is imported?
... some complex delaying of registrations ... Perhaps you can use a similar strategy.
Probably I could but it seems quite complex compared to a slightly more complete initialization of the CA via Python (rather then the late initialization via Five). Moreover, I would have to find the correct time to activate the delayed registrations. This time must be after "Products.Five.initialize" is called -- and nothing specifies the order in which products are initialized. -- Dieter
On Wednesday 10 January 2007 05:45, Martijn Faassen wrote:
I'm a bit surprised why the huge [warning] is necessary. What I understand you're saying is that you cannot use the component architecture before it's done initializing. You also assert that you need a registration to happen at Python import time. I'm curious to hear more about why you need to do this - after all, you're talking about listening to request start and end, and this happens after import time.
If you are populating the CA before ZCML is fully parsed, then: DO NOT POPULATE THE CA WHILE IMPORTING!!! This is so bad on so many levels. Dieter, your problem is just one example. Another is that unit tests have no chance to work reliably either. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Stephan Richter wrote:
If you are populating the CA before ZCML is fully parsed, then:
Why is the CA predicated on ZCML?! Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Thursday 11 January 2007 05:59, Chris Withers wrote:
Stephan Richter wrote:
If you are populating the CA before ZCML is fully parsed, then:
Why is the CA predicated on ZCML?!
Because that's how it works by default. Another way of saying this is: Collect all registrations before executing them! Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Chris Withers wrote:
Stephan Richter wrote:
If you are populating the CA before ZCML is fully parsed, then:
Why is the CA predicated on ZCML?!
It's not. And Stephan's "Collect all registrations before executing them!" isn't actually necessary either (it's just necessary for conflict resolution; grok doesn't do it, for example). The only problem lies with registrations upon import: * You can't import the module/package w/o having the registration to take place (so you can't subclass the components and have those registered instead of the orginial ones, for example) * Import happens once, so when you have unit tests that set up and clear the Component Architecture properly, you will loose all of those registrations that happened on import pretty fast. -- http://worldcookery.com -- Professional Zope documentation and training 2nd edition of Web Component Development with Zope 3 is now shipping!
Stephan Richter wrote at 2007-1-11 03:12 -0500:
... If you are populating the CA before ZCML is fully parsed, then:
DO NOT POPULATE THE CA WHILE IMPORTING!!!
This is so bad on so many levels. Dieter, your problem is just one example. Another is that unit tests have no chance to work reliably either.
How do you solve the unit test issues when the CA is initialized and populated by ZCML only? Do you fully parse the ZCML configuration file before any unit test? In my specific case, tests would make no problems (as we have only a single thread) -- but of course, you had the general case in view. -- Dieter
On Thursday 11 January 2007 16:03, Dieter Maurer wrote:
Stephan Richter wrote at 2007-1-11 03:12 -0500:
... If you are populating the CA before ZCML is fully parsed, then:
DO NOT POPULATE THE CA WHILE IMPORTING!!!
This is so bad on so many levels. Dieter, your problem is just one example. Another is that unit tests have no chance to work reliably either.
How do you solve the unit test issues when the CA is initialized and populated by ZCML only? Do you fully parse the ZCML configuration file before any unit test?
For unit tests we do not use ZCML, we use the zope.component API. In those cases, however, we still do not have side-effects from importing. The unit test case is much easier, since you only test a unit and within this units conflicts of declarations are highly unlikely.
In my specific case, tests would make no problems (as we have only a single thread) -- but of course, you had the general case in view.
Yes, I tried to make a general statement to the generall developer. My intent was not going to make a detailed discussions of all the pros and cons. After all, this was the zope@zope.org mailing list and not zope3-users or zope3-dev. ;-) Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Dieter Maurer wrote:
I tried to use Zope3 events to get informed when requests start and end.
One of our modules (the interface module to "jpype") requires such a notification for reliable work. Therefore, it tried to register the corresponding subscriptions on import of this module.
That is an uncommon approach, at least it used to be. Typically, we first have a bootstrap level and then we register things. During the bootstrap level, the Component Architecture is initialized. Things changed a bit after Zope 2.8/X3 3.0. Nowadays, the Component Architecture doesn't need any setup anymore and you can do registrations on import. I'd still recommend against that.
Unfortunately, this works very unreliably -- for the following reasons:
The CA performs by itself only extremely minimal initialization. Especially, it does not register the "Adapters" service, necessary for the registration of event subscriptions.
The full CA initialization is only performed quite late, in the call to "Products.Five.initialize". Event notification registration attempts before this time will fail (with a "ComponentLookupError: 'Adapters'"), those performed after this time will succeed.
The "Five" initialization is stupid enough that it will fail when the service registration for 'Adapters' has already been performed.
That's because Five assumes it's responsible for the setup layer. I suggest to register components through ZCML. Five will pick up configure.zcml in products automatically *after* everything has been set up properly. That's the canonical way, at least in Zope 2.8/X3 3.0
At the current state, event notifications can reliably only be defined via ZCML. This is unfortunate for our use case where the registration should be bound to the import of a given module.
I typically find it quite magic when application policy things happen at import: Never mix mechanism with policy -- Brooks' Second Law of Software Reuse Philipp P.S.: I agree with Martijn, the big "warning" is uncalled for. You're just trying to fight the semantics of the framework. -- http://worldcookery.com -- Professional Zope documentation and training 2nd edition of Web Component Development with Zope 3 is now shipping!
Philipp von Weitershausen wrote at 2007-1-10 15:40 +0100:
... Things changed a bit after Zope 2.8/X3 3.0. Nowadays, the Component Architecture doesn't need any setup anymore and you can do registrations on import.
Very good!
I'd still recommend against that.
Why?
... That's because Five assumes it's responsible for the setup layer. I suggest to register components through ZCML. Five will pick up configure.zcml in products automatically *after* everything has been set up properly.
The registration is necessary for the module *and not* any product. It might be possible that any product that uses the module registers for the event -- but it would break modularity and would be slightly inefficient.
... I typically find it quite magic when application policy things happen at import:
It is not application policy but a module requirement: *It* must be informed whenever it is used in a different thread. In Zope, this may be approximated via an information about the beginning of each request processing.
.... P.S.: I agree with Martijn, the big "warning" is uncalled for. You're just trying to fight the semantics of the framework.
That the semantics meanwhile has changed (as you note above) indicates that is has not really been right. And fighting against wrong semantics is a good thing :-) On the other hand, I agree that my fight came late as the current release already allows what I was looking for. Apparently, others already fought successfully... -- Dieter
On 1/10/07, Dieter Maurer <dieter@handshake.de> wrote:
*It* must be informed whenever it is used in a different thread.
Perhaps it could use some thread-local data to keep track of this? threading.local comes to mind. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Every sin is the result of a collaboration." --Lucius Annaeus Seneca
participants (6)
-
Chris Withers -
Dieter Maurer -
Fred Drake -
Martijn Faassen -
Philipp von Weitershausen -
Stephan Richter