How to commit a piece of data in a transaction when the transaction is being aborted later
I am dealing with a zope system where Verisign is sending a POST to our server for ecommerece purposes. Currently I am writing the entire contents of the POST from verisign to the ZODB however I have a problem that some of these POSTs are not being recorded because an error occurs later in the transaction. What I would like to do is isolate this recording from everything else. So that no matter what other error is raised that part will commit so I have a record of the conversation later. Currently I can put the logging step as the very first step in the conversation with verisign and call get_transaction().commit() (I am currently using zope 2.7 and working on the migration to zope 2.9) What would be the best way to deal with this problem? Should I just do a commit of the transaction as the very first step so that part is written and then zope will start a new transaction for the rest of the stuff done? Should I somehow tell zope to run a certain function as its own transaction? Also when I switch over to zope 2.9 how will these things need to be changed? Thanks
On Jun 20, 2006, at 2:27 PM, William Heymann wrote:
I am dealing with a zope system where Verisign is sending a POST to our server for ecommerece purposes. Currently I am writing the entire contents of the POST from verisign to the ZODB however I have a problem that some of these POSTs are not being recorded because an error occurs later in the transaction.
What I would like to do is isolate this recording from everything else. So that no matter what other error is raised that part will commit so I have a record of the conversation later.
I'd just record the data from Verisign in response to their POST (do nothing else) and come along later with a separate request to pick through the data to do postprocessing on it every so often; then the data always gets recorded and you can deal with errors in your postprocessing without needing to manage transaction state manually. ClockServer can help with the "do something every so often" aspect of this. - C
On Tuesday 20 June 2006 12:47, Chris McDonough wrote:
I'd just record the data from Verisign in response to their POST (do nothing else) and come along later with a separate request to pick through the data to do postprocessing on it every so often; then the data always gets recorded and you can deal with errors in your postprocessing without needing to manage transaction state manually. ClockServer can help with the "do something every so often" aspect of this.
That is probably a good idea for long term to change things that way and I would like to rewrite it. However right now I don't really want to rewrite the way the current system works. I had hoped I could just use get_transaction().commit() almost immediately after seeing the data from Verisign. I do like the idea of ClockServer though and I will probably start playing with it soon to see how it will work for me. Does it work with zope 2.9 also? Thanks
On Jun 20, 2006, at 2:55 PM, William Heymann wrote:
That is probably a good idea for long term to change things that way and I would like to rewrite it. However right now I don't really want to rewrite the way the current system works. I had hoped I could just use get_transaction().commit() almost immediately after seeing the data from Verisign.
Sure, you can do this, although the new spelling is "import transaction; transaction.commit()". I guess the only real problem with doing this is that you don't always know exactly what you're committing if you can't guarantee a fixed starting point as an invariant (e.g. if you kick off the method that does this *not* as a result of a request from Verisign). It might also be a little confusing for Verisign to get an error response from your server sometimes but not other times even though you record the info regardless of the eventual response, but maybe they don't attempt to retry failed requests.
I do like the idea of ClockServer though and I will probably start playing with it soon to see how it will work for me. Does it work with zope 2.9 also?
Yes. It will ship with 2.10. - C
William Heymann schrieb:
On Tuesday 20 June 2006 12:47, Chris McDonough wrote:
I'd just record the data from Verisign in response to their POST (do nothing else) and come along later with a separate request to pick through the data to do postprocessing on it every so often; then the data always gets recorded and you can deal with errors in your postprocessing without needing to manage transaction state manually. ClockServer can help with the "do something every so often" aspect of this.
That is probably a good idea for long term to change things that way and I would like to rewrite it. However right now I don't really want to rewrite the way the current system works. I had hoped I could just use get_transaction().commit() almost immediately after seeing the data from Verisign.
The problem is, there is no garanty about success of that transaction either. So if it rolls back, what do you do with verysign? Maybe a fifo logfile could solve your problem. Regards Tino
On Wednesday 21 June 2006 01:17, Tino Wildenhain wrote:
William Heymann schrieb:
That is probably a good idea for long term to change things that way and I would like to rewrite it. However right now I don't really want to rewrite the way the current system works. I had hoped I could just use get_transaction().commit() almost immediately after seeing the data from Verisign.
The problem is, there is no garanty about success of that transaction either. So if it rolls back, what do you do with verysign? Maybe a fifo logfile could solve your problem.
Actually that is not a problem. If just trying to write the data causes a failure then verisign won't ok the transaction which is the point. However what I want to happen is for the system to try and make sure as much as possible that the data that verisign sends gets written and committed before it tries to do other stuff. If that simple step should fail then there is a very seroius problem occurring and the system should not be okaying any orders. The thing with Verisign is called a silent post confirmation. Before verisign will okay a customer transaction it sends a post to my server with various details and my code will look over that and decide if it is all ok. A response code of 200 means ok, anything else means failure. Thus any uncaught error in zope will cause the transaction to fail which is the point. I just want to try and record the data that verisign sent and commit it before I try the rest so that what if something later should cause it to fail I still have a record of the commands send. Right now I am essentially writing the data to an OOBTree object.
William Heymann schrieb:
On Wednesday 21 June 2006 01:17, Tino Wildenhain wrote:
William Heymann schrieb:
That is probably a good idea for long term to change things that way and I would like to rewrite it. However right now I don't really want to rewrite the way the current system works. I had hoped I could just use get_transaction().commit() almost immediately after seeing the data from Verisign.
The problem is, there is no garanty about success of that transaction either. So if it rolls back, what do you do with verysign? Maybe a fifo logfile could solve your problem.
Actually that is not a problem. If just trying to write the data causes a failure then verisign won't ok the transaction which is the point. However what I want to happen is for the system to try and make sure as much as possible that the data that verisign sends gets written and committed before it tries to do other stuff. If that simple step should fail then there is a very seroius problem occurring and the system should not be okaying any orders.
I dont understand what you mean here. Usually the zope transaction spans the whole process (request) so if you have some tasks after your verisign remote request, they will all be covered in the transaction - so if any of them fail, verisign will get that failure message too and no false orders will be taken.
The thing with Verisign is called a silent post confirmation. Before verisign will okay a customer transaction it sends a post to my server with various details and my code will look over that and decide if it is all ok. A response code of 200 means ok, anything else means failure. Thus any uncaught error in zope will cause the transaction to fail which is the point.
Which transaction? The Verisign control post? That failing would be correct because something indeed went wrong!
I just want to try and record the data that verisign sent and commit it before I try the rest so that what if something later should cause it to fail I still have a record of the commands send. Right now I am essentially writing the data to an OOBTree object.
Well, but your essential steps indeed failed so why would you commit? How do you recover half done transactions? Regards Tino Wildenhain
----- Original Message ----- From: "William Heymann" <kosh@aesaeion.com> To: <zope@zope.org> Sent: Tuesday, June 20, 2006 2:27 PM Subject: [Zope] How to commit a piece of data in a transaction when thetransaction is being aborted later
I am dealing with a zope system where Verisign is sending a POST to our server for ecommerece purposes. Currently I am writing the entire contents of the POST from verisign to the ZODB however I have a problem that some of these POSTs are not being recorded because an error occurs later in the transaction.
What I would like to do is isolate this recording from everything else. So that no matter what other error is raised that part will commit so I have a record of the conversation later.
Currently I can put the logging step as the very first step in the conversation with verisign and call get_transaction().commit() (I am currently using zope 2.7 and working on the migration to zope 2.9)
What would be the best way to deal with this problem? Should I just do a commit of the transaction as the very first step so that part is written and then zope will start a new transaction for the rest of the stuff done? Should I somehow tell zope to run a certain function as its own transaction? Also when I switch over to zope 2.9 how will these things need to be changed?
You could write the post data to a 'non-rollbackable' sql database, then any subsequent processing errors won't impact the data you have already written to the sql db. Jonathan
William Heymann wrote:
I am dealing with a zope system where Verisign is sending a POST to our server for ecommerece purposes. Currently I am writing the entire contents of the POST from verisign to the ZODB however I have a problem that some of these POSTs are not being recorded because an error occurs later in the transaction.
What I would like to do is isolate this recording from everything else. So that no matter what other error is raised that part will commit so I have a record of the conversation later.
Currently I can put the logging step as the very first step in the conversation with verisign and call get_transaction().commit() (I am currently using zope 2.7 and working on the migration to zope 2.9)
What would be the best way to deal with this problem? Should I just do a commit of the transaction as the very first step so that part is written and then zope will start a new transaction for the rest of the stuff done? Should I somehow tell zope to run a certain function as its own transaction? Also when I switch over to zope 2.9 how will these things need to be changed?
The transaction.commit() others suggested is what I would do too. But be aware that if the second part of your transaction provokes a ConflictError (which may happen in the normal course of events), your whole request will be retried, including reinterpreting the POSTed data (which the request always saves anyway) and writing to your OOBTree etc. Florent -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
participants (5)
-
Chris McDonough -
Florent Guillaume -
Jonathan -
Tino Wildenhain -
William Heymann