Hi, If I have a DTML document for generating and tracking say order numbers, is the code rentrant/Atomic? <dtml-call "REQUEST.set('counter', counter + 1)"> <dtml-call "manage_changeProperties( counter=REQUEST['counter'] )"> <dtml-return counter > What I'm really asking is how many threads run in parallel, and how would multiple threads accessing this code react. Do I need some form of transation or locking? Thanks in anticipation David Kyte
I could be wrong here, but we did something similar, setting proprieties like this. Although the code is re-entrant what you may see, if you have high enough traffic rates, is that depending on exactly what you are doing you may get Conflicting write exceptions. Additionally, you should be aware that each manage_changeProperties will be logged in your ZODB (look at the undo list) which means that your ZODB will grow. If you have high traffic rates it will grow at an unbelievable rate (ours did). The conflicting write scenario and the writing to the ZODB also means that you will suffer (maybe minor) performance degradation. Are you using a database? Dan David Kyte wrote:
Hi,
If I have a DTML document for generating and tracking say order numbers, is the code rentrant/Atomic?
<dtml-call "REQUEST.set('counter', counter + 1)"> <dtml-call "manage_changeProperties( counter=REQUEST['counter'] )"> <dtml-return counter >
What I'm really asking is how many threads run in parallel, and how would multiple threads accessing this code react.
Do I need some form of transation or locking?
Thanks in anticipation
David Kyte
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
This is a great use for a nonundo ZODB database, especially with the new conflict resolution code in 2.3.1b2+. I'm going to be releasing a "packless" BerkeleyDB-backed nonundo implementation sometime today. I really wish somebody would write a HowTo that explains in generic terms how to use the External Mount product to mount a storage. Randy Kern wrote a great howto that touches on it (http://www.zope.org/Members/randy/ZEO-Sessions), but it's somewhat CoreSessionTracking specific. To answer the question, however, each thread in Zope maintains a single connection to the ZODB. When a commit happens (implied at the end of a request), the connection attempts to modify the database. If another thread is modifying the same object, a ConflictError will be raised and (in Zope) the request will be retried. You needn't worry about locking. You *do* however need to worry about ZODB growth and the hotspot implied by the counter. Though it's ill-documented at the moment, 2.3.1b2+ has app-level conflict resolution code that helps in this case! See http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResolution. The new class Length in the module Length.py in the BTrees package in 2.3.1b2 is an example of such a counter which makes use of conflict resolution. ----- Original Message ----- From: "Daniel Rusch" <drusch@globalcrossing.com> To: "David Kyte" <david.kyte@microamps.com> Cc: <zope@zope.org> Sent: Wednesday, March 28, 2001 10:02 AM Subject: Re: [Zope] Counter Rentrancy
I could be wrong here, but we did something similar, setting proprieties like this. Although the code is re-entrant what you may see, if you have high enough traffic rates, is that depending on exactly what you are doing you may get Conflicting write exceptions. Additionally, you should be aware that each manage_changeProperties will be logged in your ZODB (look at the undo list) which means that your ZODB will grow. If you have high traffic rates it will grow at an unbelievable rate (ours did). The conflicting write scenario and the writing to the ZODB also means that you will suffer (maybe minor) performance degradation. Are you using a database?
Dan David Kyte wrote:
Hi,
If I have a DTML document for generating and tracking say order numbers, is the code rentrant/Atomic?
<dtml-call "REQUEST.set('counter', counter + 1)"> <dtml-call "manage_changeProperties( counter=REQUEST['counter'] )"> <dtml-return counter >
What I'm really asking is how many threads run in parallel, and how would multiple threads accessing this code react.
Do I need some form of transation or locking?
Thanks in anticipation
David Kyte
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
I am still abit confused about Atomicity. In the counter code if two threads can read the counter value and bump it by one each, then great there is a lock to prevent simultaneous writes back to the property. However both threads are trying to write back N+1 not one writing N+1 and the other N+2. When the code backs off, does it back off and reread the counter? Dave -----Original Message----- From: Chris McDonough [mailto:chrism@digicool.com] Sent: 28 March 2001 17:15 To: Daniel Rusch; David Kyte Cc: zope@zope.org Subject: Re: [Zope] Counter Rentrancy This is a great use for a nonundo ZODB database, especially with the new conflict resolution code in 2.3.1b2+. I'm going to be releasing a "packless" BerkeleyDB-backed nonundo implementation sometime today. I really wish somebody would write a HowTo that explains in generic terms how to use the External Mount product to mount a storage. Randy Kern wrote a great howto that touches on it (http://www.zope.org/Members/randy/ZEO-Sessions), but it's somewhat CoreSessionTracking specific. To answer the question, however, each thread in Zope maintains a single connection to the ZODB. When a commit happens (implied at the end of a request), the connection attempts to modify the database. If another thread is modifying the same object, a ConflictError will be raised and (in Zope) the request will be retried. You needn't worry about locking. You *do* however need to worry about ZODB growth and the hotspot implied by the counter. Though it's ill-documented at the moment, 2.3.1b2+ has app-level conflict resolution code that helps in this case! See http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResolution. The new class Length in the module Length.py in the BTrees package in 2.3.1b2 is an example of such a counter which makes use of conflict resolution. ----- Original Message ----- From: "Daniel Rusch" <drusch@globalcrossing.com> To: "David Kyte" <david.kyte@microamps.com> Cc: <zope@zope.org> Sent: Wednesday, March 28, 2001 10:02 AM Subject: Re: [Zope] Counter Rentrancy
I could be wrong here, but we did something similar, setting proprieties like this. Although the code is re-entrant what you may see, if you have high enough traffic rates, is that depending on exactly what you are doing you may get Conflicting write exceptions. Additionally, you should be aware that each manage_changeProperties will be logged in your ZODB (look at the undo list) which means that your ZODB will grow. If you have high traffic rates it will grow at an unbelievable rate (ours did). The conflicting write scenario and the writing to the ZODB also means that you will suffer (maybe minor) performance degradation. Are you using a database?
Dan David Kyte wrote:
Hi,
If I have a DTML document for generating and tracking say order numbers, is the code rentrant/Atomic?
<dtml-call "REQUEST.set('counter', counter + 1)"> <dtml-call "manage_changeProperties( counter=REQUEST['counter'] )"> <dtml-return counter >
What I'm really asking is how many threads run in parallel, and how would multiple threads accessing this code react.
Do I need some form of transation or locking?
Thanks in anticipation
David Kyte
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
When it tries to commit, it will a thread retry three times. If it cant commit it will raise an error. There is also an FSCounter and a threadsafecounter that solve these problems for you and store the data on the file system preventing bloat. Both can be found on zope.org. Cheers. -- Andy McKay. ----- Original Message ----- From: "David Kyte" <david.kyte@microamps.com> To: <zope@zope.org> Sent: Wednesday, March 28, 2001 7:58 AM Subject: RE: [Zope] Counter Rentrancy
I am still abit confused about Atomicity.
In the counter code if two threads can read the counter value and bump it by one each, then great there is a lock to prevent simultaneous writes back to the property.
However both threads are trying to write back N+1 not one writing N+1 and the other N+2.
When the code backs off, does it back off and reread the counter?
Dave
-----Original Message----- From: Chris McDonough [mailto:chrism@digicool.com] Sent: 28 March 2001 17:15 To: Daniel Rusch; David Kyte Cc: zope@zope.org Subject: Re: [Zope] Counter Rentrancy
This is a great use for a nonundo ZODB database, especially with the new conflict resolution code in 2.3.1b2+. I'm going to be releasing a "packless" BerkeleyDB-backed nonundo implementation sometime today. I really wish somebody would write a HowTo that explains in generic terms
how
to use the External Mount product to mount a storage. Randy Kern wrote a great howto that touches on it (http://www.zope.org/Members/randy/ZEO-Sessions), but it's somewhat CoreSessionTracking specific.
To answer the question, however, each thread in Zope maintains a single connection to the ZODB. When a commit happens (implied at the end of a request), the connection attempts to modify the database. If another thread is modifying the same object, a ConflictError will be raised and (in Zope) the request will be retried. You needn't worry about locking. You *do* however need to worry about ZODB growth and the hotspot implied by the counter. Though it's ill-documented at the moment, 2.3.1b2+ has app-level conflict resolution code that helps in this case! See http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResolution.
The new class Length in the module Length.py in the BTrees package in 2.3.1b2 is an example of such a counter which makes use of conflict resolution.
----- Original Message ----- From: "Daniel Rusch" <drusch@globalcrossing.com> To: "David Kyte" <david.kyte@microamps.com> Cc: <zope@zope.org> Sent: Wednesday, March 28, 2001 10:02 AM Subject: Re: [Zope] Counter Rentrancy
I could be wrong here, but we did something similar, setting proprieties like this. Although the code is re-entrant what you may see, if you have high enough traffic rates, is that depending on exactly what you are doing you may get Conflicting write exceptions. Additionally, you should be aware that each manage_changeProperties will be logged in your ZODB (look at the undo list) which means that your ZODB will grow. If you have high traffic rates it will grow at an unbelievable rate (ours did). The conflicting write scenario and the writing to the ZODB also means that you will suffer (maybe minor) performance degradation. Are you using a database?
Dan David Kyte wrote:
Hi,
If I have a DTML document for generating and tracking say order numbers, is the code rentrant/Atomic?
<dtml-call "REQUEST.set('counter', counter + 1)"> <dtml-call "manage_changeProperties( counter=REQUEST['counter'] )"> <dtml-return counter >
What I'm really asking is how many threads run in parallel, and how would multiple threads accessing this code react.
Do I need some form of transation or locking?
Thanks in anticipation
David Kyte
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
participants (4)
-
Andy McKay -
Chris McDonough -
Daniel Rusch -
David Kyte