Hi guys, There's a new product available, which enables unique ids in a given context, take a look at <url:http://www.zope.org/Members/morphex/ThreadSafeCounter>. Cheers, Morten
"Morten W. Petersen" wrote:
There's a new product available, which enables unique ids in a given context, take a look at <url:http://www.zope.org/Members/morphex/ThreadSafeCounter>.
So would a counter such as: class PersistentCounter(Persistent): # create the counter def __init__(self, value=0): self._value = value # get the value of the counter without incrementing def getValue(self): return self._value # increment the counter when called, and return the new value def __call__(self, number=1): self._value = self._value + number return self._value ...not be thread safe? If not, can anyone explain how it could be made thread safe? cheers, Chris
Chris Withers wrote:
"Morten W. Petersen" wrote:
There's a new product available, which enables unique ids in a given context, take a look at <url:http://www.zope.org/Members/morphex/ThreadSafeCounter>.
So would a counter such as:
class PersistentCounter(Persistent):
# create the counter def __init__(self, value=0): self._value = value
# get the value of the counter without incrementing def getValue(self): return self._value
# increment the counter when called, and return the new value def __call__(self, number=1): self._value = self._value + number return self._value
...not be thread safe?
If not, can anyone explain how it could be made thread safe?
That's not the issue. Morten's product stores the counter's value in a file on the filesystem, and so it doesn't cause the Data.fs to grow. The trade-off is that it doesn't participate in history or undo. -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net
Steve Alexander wrote:
Chris Withers wrote:
So would a counter such as:
<snip my counter>
That's not the issue.
Does that mean my counter is thread safe?
Morten's product stores the counter's value in a file on the filesystem, and so it doesn't cause the Data.fs to grow.
Hmmm, well, my counter above wouldn't cause the data.fs to grow all that much, since it subclasses Persistent and so gets its own pickle jar. cheers, Chris
[Chris Withers] | So would a counter such as: | | class PersistentCounter(Persistent): | | # create the counter | def __init__(self, value=0): | self._value = value | | # get the value of the counter without incrementing | def getValue(self): | return self._value | | # increment the counter when called, and return the new value | def __call__(self, number=1): | self._value = self._value + number | return self._value | | ...not be thread safe? I'm not sure whether the code above is thread safe or not. ThreadSafeCounter was made because I don't know enough about Zope internals to determine whether or not the counter would return unique values, nor do I know enough about threads to be completely sure that it would, every time. File locking is a simple concept, and easy to understand. As I've understood it, two threads serving requests have a copy each of the database, and only when changes are committed are they reflected in the database. Therefore, two requests created at the same time could get an identical copy and therefore and identical value. ..so, it basically boils down to the fact that it's simple to use file locking. Cheers, Morten
On Mon, Feb 12, 2001 at 01:31:04AM +0100, Morten W. Petersen wrote:
As I've understood it, two threads serving requests have a copy each of the database, and only when changes are committed are they reflected in the database. Therefore, two requests created at the same time could get an identical copy and therefore and identical value.
The ZODB will invalidate and force a retry on one of the connections. Chris's code is threadsafe and will result in unique, sequential values. See the ZODB UML documentation for details: http://www.zope.org/Documentation/Developer/Models/ZODB/ -- Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
On Mon, 12 Feb 2001 10:27:02 +0100, Martijn Pieters <mj@digicool.com> wrote:
The ZODB will invalidate and force a retry on one of the connections. Chris's code is threadsafe and will result in unique, sequential values. **********
Unless a transaction gets retried for some other reason, when it will appear to skip a value. Toby Dickenson tdickenson@geminidataloggers.com
On Mon, Feb 12, 2001 at 02:27:24PM +0000, Toby Dickenson wrote:
On Mon, 12 Feb 2001 10:27:02 +0100, Martijn Pieters <mj@digicool.com> wrote:
The ZODB will invalidate and force a retry on one of the connections. Chris's code is threadsafe and will result in unique, sequential values. **********
Unless a transaction gets retried for some other reason, when it will appear to skip a value.
To one thread, yes. But not to the whole application. -- Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
[Morten W. Petersen] | There's a new product available, which enables unique ids in a given context, | take a look at <url:http://www.zope.org/Members/morphex/ThreadSafeCounter>. What happens if you run this with ZEO? Will the file be kept «in sync» with all ZEO Clients?
[Erik Enge] | What happens if you run this with ZEO? Will the file be kept «in | sync» with all ZEO Clients? Good point. I don't think so. It could be that it is kept in sync with one Zope instance "being responsible" and the others calling it via XML-RPC. Cheers, Morten
Morten W. Petersen wrote:
[Erik Enge]
| What happens if you run this with ZEO? Will the file be kept «in | sync» with all ZEO Clients?
Good point. I don't think so. It could be that it is kept in sync with one Zope instance "being responsible" and the others calling it via XML-RPC.
I'd thought the original point of ThreadSafeCounter was to provide a simple sequential unique values generator, without causing writes to the Data.fs. You could look at using a MountableStorage that doesn't support undo to get a similar benefit. This would also work with ZEO. Take a look in the Core Session Tracking product to see how it is done there. Also, you may find Shane Hathaway's ExternalMount product useful. -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net
[Steve Alexander] | I'd thought the original point of ThreadSafeCounter was to provide | a simple sequential unique values generator, without causing | writes to the Data.fs. Yes, that was the original intent. But having one that's safe over multiple ZEO clients is a Very Good Thing (tm). :-) | You could look at using a MountableStorage that doesn't support | undo to get a similar benefit. This would also work with ZEO. | | Take a look in the Core Session Tracking product to see how | it is done there. | | Also, you may find Shane Hathaway's ExternalMount product useful. Thanks, much appreciated. -Morten
On 10 Feb 2001, 13:01 Morten W. Petersen wrote:
There's a new product available, which enables unique ids in a given context, take a look at <url:http://www.zope.org/Members/morphex/ThreadSafeCounter>.
Doesn't even install on Windows, because it imports and uses fcntl.
From the fcntl docs: "Availability: Unix".
-- o ( Wolfgang.Strobl@gmd.de (+49 2241) 14-2394 /\ * GMD mbH #include _`\ `_<=== Schloss Birlinghoven, <std.disclaimer> __(_)/_(_)___.-._ 53754 Sankt Augustin, Germany ________________
[Wolfgang Strobl] | Doesn't even install on Windows, because it imports and uses fcntl. | | >From the fcntl docs: "Availability: Unix". Well, the download page says "Platform: Generic UNIX-like", doesn't it? -Morten
On 12 Feb 2001, 12:01 Morten W. Petersen wrote:
[Wolfgang Strobl]
| Doesn't even install on Windows, because it imports and uses fcntl. | | >From the fcntl docs: "Availability: Unix".
Well, the download page says "Platform: Generic UNIX-like", doesn't it?
Well, yes. I wouldn't have expected that kind of platform dependendy in products like AddressBook, though. -- o ( Wolfgang.Strobl@gmd.de (+49 2241) 14-2394 /\ * GMD mbH #include _`\ `_<=== Schloss Birlinghoven, <std.disclaimer> __(_)/_(_)___.-._ 53754 Sankt Augustin, Germany ________________
[Wolfgang Strobl] | Well, yes. I wouldn't have expected that kind of platform | dependendy in products like AddressBook, though. Anyways, I'm looking into ways of making the threadsafe counter platform independent. Cheers, Morten
[Morten W. Petersen] | There's a new product available, which enables unique ids in a given context, | take a look at <url:http://www.zope.org/Members/morphex/ThreadSafeCounter>. On every view - a call to index_html() - it prints '{}' to stdout. Bug?
[Erik Enge] | On every view - a call to index_html() - it prints '{}' to stdout. Bug? Forget it. My fault. *shame, shame*
[Erik Enge] | Forget it. My fault. *shame, shame* *chuckle* :-) -Morten
participants (7)
-
Chris Withers -
Erik Enge -
Martijn Pieters -
morten@esol.no -
Steve Alexander -
Toby Dickenson -
Wolfgang Strobl