post publishing hook
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times. Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying. I'd like a post-publishing hook that's called in the initial REQUEST and transaction. Open issues are: 1. what if there's an exception in the hook? I'd say log it but that's all. 2. what if there's a Conflicterror? That's tougher to deal with... The transaction has to be retried, without the user being aware of it, so it means the hook should probably be called *before* things are sent to the browser. What do you think? Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
When do you think it should be called? I just had to override zpublisher_validated_hook to do some stuff that would have been a bit easier if a different hook existed, but that might be a different use case... On Fri, 2005-04-01 at 10:48, Florent Guillaume wrote:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying.
I'd like a post-publishing hook that's called in the initial REQUEST and transaction.
Open issues are: 1. what if there's an exception in the hook? I'd say log it but that's all. 2. what if there's a Conflicterror? That's tougher to deal with... The transaction has to be retried, without the user being aware of it, so it means the hook should probably be called *before* things are sent to the browser.
What do you think?
Florent
I didn't know about that hook, but from reading the code it seems it takes place before publishing, not after. There's the recent post_traverse hook too, but that's not what I want. Florent Chris McDonough wrote:
When do you think it should be called? I just had to override zpublisher_validated_hook to do some stuff that would have been a bit easier if a different hook existed, but that might be a different use case...
On Fri, 2005-04-01 at 10:48, Florent Guillaume wrote:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying.
I'd like a post-publishing hook that's called in the initial REQUEST and transaction.
Open issues are: 1. what if there's an exception in the hook? I'd say log it but that's all. 2. what if there's a Conflicterror? That's tougher to deal with... The transaction has to be retried, without the user being aware of it, so it means the hook should probably be called *before* things are sent to the browser.
What do you think?
Florent
-- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Chris McDonough <chrism@plope.com> wrote:
When do you think it should be called?
I should have answered that clearly: I'd put it in ZPublisher.publish, like this: result=mapply(object, request.args, request, call_object,1, missing_name, dont_publish_class, request, bind=1) # INSERT HOOK CALL HERE if result is not response: response.setBody(result) # OR MAYBE HERE but above is probably better if transactions_manager: transactions_manager.commit() Florent
I just had to override zpublisher_validated_hook to do some stuff that would have been a bit easier if a different hook existed, but that might be a different use case...
On Fri, 2005-04-01 at 10:48, Florent Guillaume wrote:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying.
I'd like a post-publishing hook that's called in the initial REQUEST and transaction.
Open issues are: 1. what if there's an exception in the hook? I'd say log it but that's all. 2. what if there's a Conflicterror? That's tougher to deal with... The transaction has to be retried, without the user being aware of it, so it means the hook should probably be called *before* things are sent to the browser.
What do you think?
Florent
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
-- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Florent Guillaume wrote:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Using a post publishing hook for this use case isn't a good idea - IMO. The Publisher has nothing to with indexing. It's a completly different part of the architecture. You want to bind some actions to the transaction system because you want to execute some code at the end of the first phase of the 2 phase commit. That's a completly different story than the post publishing hook. Hooking into the transaction machinery of ZODB is very easy. Write your own DateManager implementing the IDateManager interface and register it into the current transaction using get_transaction.register(DM). For your use case overwrite the tpc_vote() method of the data manager to do the indexing at the boundary between the two phases. Jens Vagenpohl's mail host and Chris McD's blob have example code for you. Christian
Christian Heimes <heimes@faho.rwth-aachen.de> wrote:
Florent Guillaume wrote:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Using a post publishing hook for this use case isn't a good idea - IMO. The Publisher has nothing to with indexing. It's a completly different part of the architecture.
You want to bind some actions to the transaction system because you want to execute some code at the end of the first phase of the 2 phase commit. That's a completly different story than the post publishing hook. Hooking into the transaction machinery of ZODB is very easy. Write your own DateManager implementing the IDateManager interface and register it into the current transaction using get_transaction.register(DM). [note: s/IDateManager/IDataManager/ above]
For your use case overwrite the tpc_vote() method of the data manager to do the indexing at the boundary between the two phases.
Jens Vagenpohl's mail host and Chris McD's blob have example code for you.
Ok, Tres also recommended something like that. I was a bit concerned that there would be problems in case of conflicts, or if there was a need to tweak the body returned to the user, but after reading transaction's code and MailDropHost and re-re-reading Publisher.py, I'm confident I can do everything I need. Thanks a lot, this together with events will make things much easier in our framework. Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Christian Heimes wrote at 2005-4-1 22:28 +0200:
... You want to bind some actions to the transaction system because you want to execute some code at the end of the first phase of the 2 phase commit.
I do not think that Florent wants to execute things at the end or in the first commit phase. Instead, he wants to execute something *BEFORE* this phase.
Hooking into the transaction machinery of ZODB is very easy. Write your own DateManager implementing the IDateManager interface and register it into the current transaction using get_transaction.register(DM).
For your use case overwrite the tpc_vote() method of the data manager to do the indexing at the boundary between the two phases.
This will not work for Florent's use case, at least not in the current ZODB (for Zope 2.7). Reason: The transaction freezes the set of objects it processes in the commit *BEFORE* the first commit phase. Therefore, no "tpc_vote" is allowed to modify any persistent object. Failure to do so leads to extremely nasty persistency errors (the object is marked changed but no transaction knows about it -- no change to this object in the future will get persisted until a restart).
Jens Vagenpohl's mail host and Chris McD's blob have example code for you.
But they do not change persistent objects (at least, I hope so). -- Dieter
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Dieter Maurer wrote:
Christian Heimes wrote at 2005-4-1 22:28 +0200:
... You want to bind some actions to the transaction system because you want to execute some code at the end of the first phase of the 2 phase commit.
I do not think that Florent wants to execute things at the end or in the first commit phase. Instead, he wants to execute something *BEFORE* this phase.
That was the initial idea in our case.
Hooking into the transaction machinery of ZODB is very easy. Write your own DateManager implementing the IDateManager interface and register it into the current transaction using get_transaction.register(DM).
For your use case overwrite the tpc_vote() method of the data manager to do the indexing at the boundary between the two phases.
This will not work for Florent's use case, at least not in the current ZODB (for Zope 2.7).
Reason:
The transaction freezes the set of objects it processes in the commit *BEFORE* the first commit phase.
Therefore, no "tpc_vote" is allowed to modify any persistent object. Failure to do so leads to extremely nasty persistency errors (the object is marked changed but no transaction knows about it -- no change to this object in the future will get persisted until a restart).
I've been implementing a IndexationManager hook doing the job for CPS but I get the error your talking about... Thanks Dieter for the explanation. Your mail just arrived when I was trying to understand the persistency problems :( I got POSKeyError, Unauthorized (at catalog level while indexing and trying to get object attributs) or KeyError (Connection.setState()) exceptions randomly as well. Is there a way to hook before the first transaction commit phase ? J. - -- Julien Anguenot | Nuxeo R&D (Paris, France) CPS Plateform : http://www.cps-project.org mail: anguenot at nuxeo.com; tel: +33 (0) 6 72 57 57 66 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFCUFReGhoG8MxZ/pIRAi2jAJ4hXTAJhqJB8czSkQpVCCBEKzWXtACfXA/m S4JfeBF+TWweLWPSoxobPEE= =8zPL -----END PGP SIGNATURE-----
Dieter Maurer <dieter@handshake.de> wrote:
Christian Heimes wrote at 2005-4-1 22:28 +0200:
... You want to bind some actions to the transaction system because you want to execute some code at the end of the first phase of the 2 phase commit.
I do not think that Florent wants to execute things at the end or in the first commit phase. Instead, he wants to execute something *BEFORE* this phase.
Yes, wherever it's inserted, I want to do additional processing before everything is finalized, and in some cases I want to be able to abort the transaction, and to create and change objects in the ZODB.
Hooking into the transaction machinery of ZODB is very easy. Write your own DateManager implementing the IDateManager interface and register it into the current transaction using get_transaction.register(DM).
For your use case overwrite the tpc_vote() method of the data manager to do the indexing at the boundary between the two phases.
This will not work for Florent's use case, at least not in the current ZODB (for Zope 2.7).
Reason:
The transaction freezes the set of objects it processes in the commit *BEFORE* the first commit phase.
Therefore, no "tpc_vote" is allowed to modify any persistent object. Failure to do so leads to extremely nasty persistency errors (the object is marked changed but no transaction knows about it -- no change to this object in the future will get persisted until a restart).
Ah, that was what I was afraid of, but I was assured it was not the case. I'll have to reread that transaction code... and maybe readd the low-tech post-publishing hook I was thinking of. Thanks Dieter, Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Florent Guillaume wrote at 2005-4-1 17:48 +0200:
... Open issues are: 1. what if there's an exception in the hook? I'd say log it but that's all.
I prefer: fail in the usual way.
2. what if there's a Conflicterror? That's tougher to deal with... The transaction has to be retried, without the user being aware of it, so it means the hook should probably be called *before* things are sent to the browser.
As you do it in the same transaction, you usually do it before anything is sent to the browser (unless someone uses "RESPONSE.write"). I would handle the post publishing hook identical to the main request action. -- Dieter
Hi, Are your talking about a short thing for 2.7? In 2.8 I wonder if the Publisher could start using e.g. Zope 3 events. Are there events for this around in Z3 already? This would be a good way of improving transition to Z3 in a compatible manner, if both systems e.g. publisher modules would spit out the same events for handling those cases. Am Freitag, den 01.04.2005, 17:48 +0200 schrieb Florent Guillaume:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Christian -- gocept gmbh & co. kg - schalaunische str. 6 - 06366 koethen - germany www.gocept.com - ct@gocept.com - phone +49 3496 30 99 112 - fax +49 3496 30 99 118 - zope and plone consulting and development
Christian Theune wrote:
Florent Guillaume:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times. Are your talking about a short thing for 2.7?
Yes we need it for CPS which currently works with 2.7 (no customer is ready to move to 2.8 yet).
In 2.8 I wonder if the Publisher could start using e.g. Zope 3 events. Are there events for this around in Z3 already? This would be a good way of improving transition to Z3 in a compatible manner, if both systems e.g. publisher modules would spit out the same events for handling those cases.
Yes events are the way to go. For 2.7 we have our (CPS) event system, and Julien has experimented with patching Transaction.py's commit and abort to send appropriate events. Then the rest is just a matter of subscribing and reacting to them. I totally agree that if 2.8 could use standardized events from Zope 3 that would be awesome. Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Am Montag, den 04.04.2005, 13:39 +0200 schrieb Florent Guillaume:
Yes we need it for CPS which currently works with 2.7 (no customer is ready to move to 2.8 yet).
Neither is 2.8. ;) -- gocept gmbh & co. kg - schalaunische str. 6 - 06366 koethen - germany www.gocept.com - ct@gocept.com - phone +49 3496 30 99 112 - fax +49 3496 30 99 118 - zope and plone consulting and development
Florent Guillaume wrote:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying.
I'd like a post-publishing hook that's called in the initial REQUEST and transaction.
I haven't been folowing this thread, so I asked Gary what it was about. :) Based on that, I'd like to suggest: There are two possibilities: 1. A post publishing hook. I think this would be appropriate in the case where you really want to augment the publishing process. For example, I hpe someday to use something like this to provide another way (other than metal) to provide standard look and feel. Unfortunately, I think there are a lot of open issues, at least in my mind, about how something like this should work. 2. A pre-commit hook. I think your use case fits this much better. We've thought about this for some time. In fact, I've talked about it enough that I thought we already had this, but we don't. I think this is the way to go for your use case. I'll post a proposal on the zodb-dev list. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Jim Fulton wrote:
Florent Guillaume wrote:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying.
I'd like a post-publishing hook that's called in the initial REQUEST and transaction.
I haven't been folowing this thread, so I asked Gary what it was about. :) Based on that, I'd like to suggest:
There are two possibilities:
1. A post publishing hook. I think this would be appropriate in the case where you really want to augment the publishing process. For example, I hpe someday to use something like this to provide another way (other than metal) to provide standard look and feel.
Unfortunately, I think there are a lot of open issues, at least in my mind, about how something like this should work.
What I had in mind was, just after Publish.publish calls result = mapply(object, request.args, request, ...) add: if hasattr(request, 'runPostPublishingHooks'): result = request.runPostPublishingHooks(result, request, response) And a simple system for registering hooks. Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Florent Guillaume wrote:
Jim Fulton wrote:
Florent Guillaume wrote:
I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times.
Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying.
I'd like a post-publishing hook that's called in the initial REQUEST and transaction.
I haven't been folowing this thread, so I asked Gary what it was about. :) Based on that, I'd like to suggest:
There are two possibilities:
1. A post publishing hook. I think this would be appropriate in the case where you really want to augment the publishing process. For example, I hpe someday to use something like this to provide another way (other than metal) to provide standard look and feel.
Unfortunately, I think there are a lot of open issues, at least in my mind, about how something like this should work.
What I had in mind was, just after Publish.publish calls
result = mapply(object, request.args, request, ...)
add:
if hasattr(request, 'runPostPublishingHooks'): result = request.runPostPublishingHooks(result, request, response)
And a simple system for registering hooks.
I'd rather not take the time to discuss this now, since we seem to be on track with the transaction idea, which I still think is a better fit. If you really want to discuss this now, I'll respond, although maybe not today. :) Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
participants (7)
-
Chris McDonough -
Christian Heimes -
Christian Theune -
Dieter Maurer -
Florent Guillaume -
Jim Fulton -
Julien Anguenot