[Zope-CMF] Re: cmfuid

[Chris McDonough]

On Mon, 2004-11-22 at 11:31, Gregoire Weber wrote:
> Hi Chris,
> 
> >> My advice is to just replace 'portal_uidgenerator' with your own UID 
> >> or UUID generator implementation.
> >
> >We should probably fix the default one because it doesn't guarantee
> >uniqueness of ids.  The simplest fix would be to not use a Length object
> >as the counter but to use a subclass of Length that returned false from
> >its _p_independent method.  This would probably work for most people,
> >although probably not very well for applications that generated lots of
> >uids simultaneously.
> 
> Ok, I understand the issue in general. But in this case BTree-Length does 
> a read of 'value' and tries to write to the same 'value' attribute after
> having incremented it. No other variables are used to calculate the result 
> than 'value'. So shouldn't read conflicts be tracked by the write conflicts
> anyway?

Here's what happens in the pathological case (broken down by "time
units):

time 0: counter is at 0

time 1: thread 1 changes the counter during a new uid call,
        the generated uid is 1

time 1: thread 2 changes the counter during a new uid call
        the generated uid is 1

(note that no conflicts have happened yet, write conflicts are only
raised at commit time and read conflicts don't happen because the Length
object is _p_independent)

time 2: thread 1 commits

time 3: thread 2 commits, but commit generates a write conflict
        due to thread 1 being generated beforehand.  The write
        conflict for the counter is resolved and it is set to 2.

Note that the counter is indeed correct (it's now 2, the next uid handed
out will be 3) but we've handed out the uid "1" twice.  We resolved the
write conflict on the Length object at commit time but it didn't help
us.  Both threads committed after giving two different callers the same
uid, so we presumably now have two objects with the same "uid" value,
which is not desirable.

Making _p_independent of the Length object return false will cause a
read conflict to be generated at the time of uid generation (during
"change") if two threads ask for a uid simultaneously.  This has the
effect at least under Zope 2.7 of causing actually unique ids to be
generated.  This doesn't work under 2.8.  Under 2.8, MVCC begins to kick
in and we have the same problem again even if we override
_p_independent.

The very simplest thing to do here is to just use a plain integer
counter rather than a Length object, FWIW.  It's the only simple
solution that works for 2.7 and 2.8, probably at significant expense due
to retries.  Less simple strategies exist to generate uids using the
database, like the strategy used by the catalog to generate "rids" that
do guarantee uniqueness, but the code becomes much less aesthetically
pleasing.

> I'm not the ZODB expert here, so I probably miss some point. Based on
> my deep experience in C based real time system software I don't see a 
> problem.
> 
> Reading the comment of '_p_independent', I see there may be a problem,
> but I don't understand it really.
> 
> May you explain or somebody different? E.g. Tres?
> 
> If there is a real problem I would appreciate we correct the issue/bug.

I think it is a real problem, but I'm not sure of the best way to fix
it.  I'd just use a probabilistic generator but Tres doesn't like them,
I think.  I don't yet understand why, but I'm sure there's a good reason
(Tres is right about 99.3% of the time ;-).

> >> By the way: It would be cool if someone could implement <http://www.ietf.org/internet-drafts/draft-mealling-uuid-urn-03.txt> (hint, hint!). Actually I think this should be a python lib ...
> >
> >I have the beginnings of such a thing at
> >http://www.plope.com/software/uuidgen/view .  Not really, because it
> >doesn't follow the string formatting rules of the spec and it only does
> >"Version 4" UUIDs (without a version bit set) but I guess it's a start.
> 
> Does ist work also under Win32?

No, although presumably it would be pretty simple to make it do so, I'd
just steal the code from Python 2.4.

> I would propose here (after having corrected the counter issue) to add 
> a new generator tool which users can use to replace the standard one.

I'm not sure we need two default id generators, but I'll do whatever
anybody wants done.

- C