[ZODB-Dev] new-style class status

Tim Peters tim at zope.com
Sun Oct 19 22:44:12 EDT 2003


[John Belmonte]
> ...
> I did try zodb33-devel-branch after my last email and it's working
> great, although I'm too early in development to have anything to
> stress-test the system.

That branch is more ZODB3 than ZODB4, so should be pretty safe provided you
stick to the ZODB3 parts of it <wink>.  It's missing a number of critical
ZEO bugfixes in the most recent releases of 3.1 and 3.2.  As I recall, the
hardest part of zodb33-devel-branch so far was teaching the ZODB3 cache to
cooperate with Python's cyclic garbage collection (the former uses a funky
C-level "weak cache" that lies about reference counts, while the latter
relies on truthful refcounts; life gets delicate where they intersect).

> Of course I'm wary about having my app depend on a feature branch that
> may never make it back to the trunk.

I see Jim Fulton checked in some new stuff on that branch today (Sunday), so
it's got his current attention.

> Unfortunately, I've unwittingly done just that-- I didn't realize that
> extension class comparison operators were broken until after the class
> that used them was ingrained into my app.

User-defined comparison in Python is complex, especially when comparand
types are mixed, to the extent that "broken" is sometimes arguable.  We've
seen evidence here in the past that ExtensionClass adds more complications,
to the point where I gave up trying to understand it.

> I can see needing MVCC as well.

That's a scalability thing more than anything else.  Frequent read conflicts
become inevitable as the number of concurrent readers and writers increase;
MVCC aims at eliminating read conflicts.

> It seems like it would be very hard to verify that a program is
> correct if I have to worry about it making a bad decision due to
> seeing an incoherent set of database objects (that is, in the face of
> inter-object invariant violations).

MVCC isn't needed to provide consistency.  You get a ReadConflictError
without MVCC if you try to read data that's been changed by another
transaction since the time your current transaction started (with MVCC, you
get the data as of the time your current transaction started).  Then you
should abort your current transaction and try again.  Under 3.1 it's
possible to get in trouble by ignoring ReadConflictError (e.g., swallowing
it in a bare "except:" clause); under 3.2 a transaction refuses to commit if
a ReadConflictError occurred during it.

> Then I'm also in need of multiple monotonic ID generators, which
> doesn't look to be going into ZODB at all.

I doubt Zope Corp will fund work on that (since it hasn't so far, outside
gimmicks of this nature Zope uses).  If other parties do the work (API
design, debate, coding, porting, tests, docs), though, I don't expect Zope
Corp would oppose adding it.

>> One thing we need is a reimplementation of ExtensionClass as a Python
>> new-style class.  The current ExtensionClass implementation is much
>> more complicated than necessary now that Python makes far fewer
>> distinctions between types and classes, and seems essentially
>> unmaintainable to me. ExtensionClass in some API-compatible form has
>> to continue to survive, for compatibility with applications already
>> using ExtensionClass.  I know Jim (Fulton) looked at this some
>> Friday, but don't know how far he got.

> Is this a precondition for having new-style classes in ZODB or is it
> independent?

Jim makes the rules here; I know he thinks it's important (and checked in
work toward this end Sunday).

> Perhaps I could help keep the new-style class branch in sync as 3.3
> goes through its milestones, so that it will be ready to go for
> version 3.4.

Help would be most welcome!  3.2 final was pretty close to the ZODB3 trunk
at the time, and zodb33-devel-branch came off the trunk but months before
3.2 final.  They're not intractably far apart yet.




More information about the ZODB-Dev mailing list