[Zodb-checkins] SVN: ZODB/branches/3.4/src/ Merge the ZODB part of
Zope/branches/jim-fix-zclasses.
Tim Peters
tim.one at comcast.net
Wed Apr 6 18:35:06 EDT 2005
Log message for revision 29891:
Merge the ZODB part of Zope/branches/jim-fix-zclasses.
This doesn't have a lot of visible effect on standalone ZODB use, but
is important for Zope 2.8.
Here are checkin msgs from the branch relating to ZODB code:
r29872 | jim | 2005-04-04 07:04:39 -0400 (Mon, 04 Apr 2005) | 3 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/serialize.py
For instances of persistent classes, save a class-module/clas-name
tuple if the class has a non-empty module string.
------------------------------------------------------------------------
r29871 | jim | 2005-04-04 07:04:33 -0400 (Mon, 04 Apr 2005) | 6 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/Connection.py
Changed sub-transaction abort code to not invalidate created objects.
There's really no point, because created objects will be unreachable
after invalidating old objects. Also, if a created object is
non-ghostifiable, it will try to load it's state again, and then
either it would fail or it would load non-committed state.
------------------------------------------------------------------------
r29870 | jim | 2005-04-04 07:04:27 -0400 (Mon, 04 Apr 2005) | 3 lines
Changed paths:
A /Zope/branches/jim-fix-zclasses/lib/python/ZODB/persistentclass.py
A /Zope/branches/jim-fix-zclasses/lib/python/ZODB/persistentclass.txt
A /Zope/branches/jim-fix-zclasses/lib/python/ZODB/tests/testpersistentclass.py
Added ZClass-independent test of (and possible base class for)
persistent-class support machinery.
------------------------------------------------------------------------
r29867 | jim | 2005-04-04 07:03:48 -0400 (Mon, 04 Apr 2005) | 2 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/transaction/_manager.py
Added missing arguments to commit and abort.
------------------------------------------------------------------------
r29774 | jim | 2005-04-01 06:24:27 -0500 (Fri, 01 Apr 2005) | 3 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/transaction/__init__.py
Changed to use methods of a threaded manager directly, rather than
through wrapper functions.
------------------------------------------------------------------------
r29773 | jim | 2005-04-01 06:24:25 -0500 (Fri, 01 Apr 2005) | 3 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/transaction/_manager.py
Added commit and abort methods to transaction managers.
This makes direcr use of managers simpler.
------------------------------------------------------------------------
r29101 | jim | 2005-02-10 07:44:52 -0500 (Thu, 10 Feb 2005) | 2 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/Connection.py
Removed an unused attribute.
------------------------------------------------------------------------
r29100 | jim | 2005-02-10 07:42:35 -0500 (Thu, 10 Feb 2005) | 3 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/transaction/_transaction.py
Changed an XXX comment to an ordinary comment, explaining the relevent
issue.
------------------------------------------------------------------------
r29099 | jim | 2005-02-10 07:41:58 -0500 (Thu, 10 Feb 2005) | 3 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/Connection.py
Changed some XXX comments to ordinary comments explaining the relevent
issues.
------------------------------------------------------------------------
r29071 | jim | 2005-02-07 07:36:05 -0500 (Mon, 07 Feb 2005) | 3 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/transaction/_transaction.py
Added a sanity check to avoid registration of objects without a
manager. (Perhaps this should be an assert.)
------------------------------------------------------------------------
r29067 | jim | 2005-02-07 07:35:56 -0500 (Mon, 07 Feb 2005) | 15 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/Connection.py
Changed the strategy for handling invalidation of classes.
No-longer use setklassstate. Instead, just call _p_invalidate, as
with any other object. This changes didn't break any tests, so I
assume that this was untested. :(
Changed the strategy for invalidating objects. Non-ghostifiable
objects will load their state when they are invalidated. We have to
worry about other invalidations that come in while this is happening.
See the comment in _flush_invalidations.
We need to force all invalidations to take this into account, by going
through _flush_invalidations. I haven't done this yet, but I have
left some XXX comments in places where it needs to be done to remind
myself that this needs to be done.
------------------------------------------------------------------------
r29066 | jim | 2005-02-07 07:35:54 -0500 (Mon, 07 Feb 2005) | 8 lines
Changed paths:
M /Zope/branches/jim-fix-zclasses/lib/python/persistent/cPickleCache.c
Changed the strategy for handling invalidation of classes.
No-longer use setklassstate. Instead, just call _p_invalidate, as
with any other object. This changes didn't break any tests, so I
assume that this was untested. :(
Change invalidation to not swallow errors. (Swallowing errors here was a
travesty!)
Changed:
U ZODB/branches/3.4/src/Persistence/tests/test_ExtensionClass.py
U ZODB/branches/3.4/src/Persistence/tests/test_mapping.py
U ZODB/branches/3.4/src/ZODB/Connection.py
A ZODB/branches/3.4/src/ZODB/persistentclass.py
A ZODB/branches/3.4/src/ZODB/persistentclass.txt
U ZODB/branches/3.4/src/ZODB/serialize.py
U ZODB/branches/3.4/src/ZODB/tests/sampledm.py
U ZODB/branches/3.4/src/ZODB/tests/testBroken.py
U ZODB/branches/3.4/src/ZODB/tests/testConnection.py
U ZODB/branches/3.4/src/ZODB/tests/testFileStorage.py
U ZODB/branches/3.4/src/ZODB/tests/testSerialize.py
U ZODB/branches/3.4/src/ZODB/tests/testSubTransaction.py
U ZODB/branches/3.4/src/ZODB/tests/test_cache.py
U ZODB/branches/3.4/src/ZODB/tests/test_datamanageradapter.py
U ZODB/branches/3.4/src/ZODB/tests/testmvcc.py
A ZODB/branches/3.4/src/ZODB/tests/testpersistentclass.py
U ZODB/branches/3.4/src/persistent/cPickleCache.c
U ZODB/branches/3.4/src/persistent/tests/test_PickleCache.py
U ZODB/branches/3.4/src/persistent/tests/test_overriding_attrs.py
U ZODB/branches/3.4/src/persistent/tests/test_persistent.py
U ZODB/branches/3.4/src/persistent/tests/test_pickle.py
U ZODB/branches/3.4/src/persistent/tests/test_wref.py
U ZODB/branches/3.4/src/transaction/__init__.py
U ZODB/branches/3.4/src/transaction/_manager.py
U ZODB/branches/3.4/src/transaction/_transaction.py
U ZODB/branches/3.4/src/transaction/tests/test_SampleDataManager.py
U ZODB/branches/3.4/src/transaction/tests/test_register_compat.py
U ZODB/branches/3.4/src/transaction/tests/test_transaction.py
U ZODB/branches/3.4/src/transaction/tests/test_util.py
-=-
Modified: ZODB/branches/3.4/src/Persistence/tests/test_ExtensionClass.py
===================================================================
--- ZODB/branches/3.4/src/Persistence/tests/test_ExtensionClass.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/Persistence/tests/test_ExtensionClass.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -16,7 +16,7 @@
$Id$
"""
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
import pickle
from Persistence import Persistent
Modified: ZODB/branches/3.4/src/Persistence/tests/test_mapping.py
===================================================================
--- ZODB/branches/3.4/src/Persistence/tests/test_mapping.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/Persistence/tests/test_mapping.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -15,7 +15,7 @@
$Id$
"""
import unittest
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
from Persistence import PersistentMapping
def test_basic_functionality():
Modified: ZODB/branches/3.4/src/ZODB/Connection.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/Connection.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/Connection.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -120,9 +120,9 @@
# will execute atomically by virtue of the GIL. But some storage
# might generate oids where hash or compare invokes Python code. In
# that case, the GIL can't save us.
+
self._inv_lock = threading.Lock()
self._invalidated = d = {}
- self._invalid = d.has_key
# We intend to prevent committing a transaction in which
# ReadConflictError occurs. _conflicts is the set of oids that
@@ -221,6 +221,26 @@
del obj._p_jar
del obj._p_oid
else:
+
+ # Note: If we invalidate a non-ghostifiable object
+ # (i.e. a persistent class), the object will
+ # immediately reread it's state. That means that the
+ # following call could result in a call to
+ # self.setstate, which, of course, must suceed.
+ # In general, it would be better if the read could be
+ # delayed until the start of the next transaction. If
+ # we read at the end of a transaction and if the
+ # object was invalidated during this transaction, then
+ # we'll read non-current data, which we'll discard
+ # later in transaction finalization. Unfortnately, we
+ # can only delay the read if this abort corresponds to
+ # a top-level-transaction abort. We can't tell if
+ # this is a top-level-transaction abort, so we have to
+ # go ahead and invalidate now. Fortunately, it's
+ # pretty unlikely that the object we are invalidating
+ # was invalidated by another thread, so the risk of a
+ # reread is pretty low.
+
self._cache.invalidate(oid)
self._tpc_cleanup()
@@ -387,6 +407,22 @@
self._storage = self._tmp
self._tmp = None
+ # Note: If we invalidate a non-ghostifiable object (i.e. a
+ # persistent class), the object will immediately reread it's
+ # state. That means that the following call could result in a
+ # call to self.setstate, which, of course, must succeed. In
+ # general, it would be better if the read could be delayed
+ # until the start of the next transaction. If we read at the
+ # end of a transaction and if the object was invalidated
+ # during this transaction, then we'll read non-current data,
+ # which we'll discard later in transaction finalization. We
+ # could, theoretically queue this invalidation by calling
+ # self.invalidate. Unfortunately, attempts to make that
+ # change resulted in mysterious test failures. It's pretty
+ # unlikely that the object we are invalidating was invalidated
+ # by another thread, so the risk of a reread is pretty low.
+ # It's really not worth the effort to pursue this.
+
self._cache.invalidate(src._index.keys())
self._invalidate_creating(src._creating)
@@ -415,12 +451,44 @@
def _flush_invalidations(self):
self._inv_lock.acquire()
try:
- self._cache.invalidate(self._invalidated)
- self._invalidated.clear()
+
+ # Non-ghostifiable objects may need to read when they are
+ # invalidated, so, we'll quickly just replace the
+ # invalidating dict with a new one. We'll then process
+ # the invalidations after freeing the lock *and* after
+ # reseting the time. This means that invalidations will
+ # happen after the start of the transactions. They are
+ # subject to conflict errors and to reading old data,
+
+ # TODO: There is a potential problem lurking for persistent
+ # classes. Suppose we have an invlidation of a persistent
+ # class and of an instance. If the instance is
+ # invalidated first and if the invalidation logic uses
+ # data read from the class, then the invalidation could
+ # be performed with state data. Or, suppose that there
+ # are instances of the class that are freed as a result of
+ # invalidating some object. Perhaps code in their __del__
+ # uses class data. Really, the only way to properly fix
+ # this is to, in fact, make classes ghostifiable. Then
+ # we'd have to reimplement attribute lookup to check the
+ # class state and, if necessary, activate the class. It's
+ # much worse than that though, because we'd also need to
+ # deal with slots. When a class is ghostified, we'd need
+ # to replace all of the slot operations with versions that
+ # reloaded the object when caled. It's hard to say which
+ # is better for worse. For now, it seems the risk of
+ # using a class while objects are being invalidated seems
+ # small enough t be acceptable.
+
+ invalidated = self._invalidated
+ self._invalidated = {}
self._txn_time = None
finally:
self._inv_lock.release()
- # Now is a good time to collect some garbage
+
+ self._cache.invalidate(invalidated)
+
+ # Now is a good time to collect some garbage.
self._cache.incrgc()
def root(self):
@@ -532,10 +600,26 @@
self._tpc_cleanup()
def tpc_abort(self, transaction):
- """Abort a transaction."""
if self._import:
self._import = None
self._storage.tpc_abort(transaction)
+
+ # Note: If we invalidate a non-justifiable object (i.e. a
+ # persistent class), the object will immediately reread it's
+ # state. That means that the following call could result in a
+ # call to self.setstate, which, of course, must succeed. In
+ # general, it would be better if the read could be delayed
+ # until the start of the next transaction. If we read at the
+ # end of a transaction and if the object was invalidated
+ # during this transaction, then we'll read non-current data,
+ # which we'll discard later in transaction finalization. We
+ # could, theoretically queue this invalidation by calling
+ # self.invalidate. Unfortunately, attempts to make that
+ # change resulted in mysterious test failures. It's pretty
+ # unlikely that the object we are invalidating was invalidated
+ # by another thread, so the risk of a reread is pretty low.
+ # It's really not worth the effort to pursue this.
+
self._cache.invalidate(self._modified)
self._invalidate_creating()
while self._added:
@@ -630,7 +714,9 @@
# because we have to check again after the load anyway.
if (obj._p_oid in self._invalidated
- and not myhasattr(obj, "_p_independent")):
+ and not myhasattr(obj, "_p_independent")
+ and not self._invalidated
+ ):
# If the object has _p_independent(), we will handle it below.
self._load_before_or_conflict(obj)
return
@@ -913,4 +999,3 @@
if dt is not DEPRECATED_ARGUMENT:
deprecated36("cacheMinimize() dt= is ignored.")
self._cache.minimize()
-
Copied: ZODB/branches/3.4/src/ZODB/persistentclass.py (from rev 29890, Zope/branches/jim-fix-zclasses/lib/python/ZODB/persistentclass.py)
Property changes on: ZODB/branches/3.4/src/ZODB/persistentclass.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Copied: ZODB/branches/3.4/src/ZODB/persistentclass.txt (from rev 29890, Zope/branches/jim-fix-zclasses/lib/python/ZODB/persistentclass.txt)
Property changes on: ZODB/branches/3.4/src/ZODB/persistentclass.txt
___________________________________________________________________
Name: svn:eol-style
+ native
Modified: ZODB/branches/3.4/src/ZODB/serialize.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/serialize.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/serialize.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -32,14 +32,19 @@
The class description can be in a variety of formats, in part to
provide backwards compatibility with earlier versions of Zope. The
-two current formats for class description are:
+four current formats for class description are:
1. type(obj)
2. type(obj), obj.__getnewargs__()
+ 3. (module name, class name), None
+ 7. (module name, class name), obj.__getnewargs__()
The second of these options is used if the object has a __getnewargs__()
method. It is intended to support objects like persistent classes that have
-custom C layouts that are determined by arguments to __new__().
+custom C layouts that are determined by arguments to __new__(). The
+third and fourth (#3 & #7) apply to instances of a persistent class (which
+means the class itself is persistent, not that it's a subclass of
+Persistent).
The type object is usually stored using the standard pickle mechanism, which
involves the pickle GLOBAL opcode (giving the type's module and name as
@@ -59,17 +64,17 @@
Earlier versions of Zope supported several other kinds of class
descriptions. The current serialization code reads these descriptions, but
-does not write them. The four earlier formats are:
+does not write them. The three earlier formats are:
- 3. (module name, class name), None
4. (module name, class name), __getinitargs__()
5. class, None
6. class, __getinitargs__()
Formats 4 and 6 are used only if the class defines a __getinitargs__()
-method. Formats 5 and 6 are used if the class does not have a __module__
-attribute (I'm not sure when this applies, but I think it occurs for some
-but not all ZClasses).
+method, but we really can't tell them apart from formats 7 and 2
+(respectively). Formats 5 and 6 are used if the class does not have a
+__module__ attribute (I'm not sure when this applies, but I think it occurs
+for some but not all ZClasses).
Persistent references
@@ -100,6 +105,8 @@
from ZODB.broken import Broken
from ZODB.POSException import InvalidObjectReference
+_oidtypes = str, type(None)
+
# Might to update or redo coptimizations to reflect weakrefs:
# from ZODB.coptimizations import new_persistent_id
@@ -147,10 +154,10 @@
... _p_jar = jar
>>> writer = ObjectWriter(O)
- Normally, object references include the oid and a cached
- reference to the class. Having the class available allows
- fast creation of the ghost, avoiding requiring an additional
- database lookup.
+ Normally, object references include the oid and a cached named
+ reference to the class. Having the class information
+ available allows fast creation of the ghost, avoiding
+ requiring an additional database lookup.
>>> bob = P('bob')
>>> oid, cls = writer.persistent_id(bob)
@@ -282,8 +289,16 @@
# It's possible that __getnewargs__ is degenerate and
# returns (), but we don't want to have to deghostify
# the object to find out.
+
+ # Note that this has the odd effect that, if the class has
+ # __getnewargs__ of its own, we'll lose the optimization
+ # of caching the class info.
+
return oid
+ # Note that we never get here for persistent classes.
+ # We'll use driect refs for normal classes.
+
return oid, klass
def serialize(self, obj):
@@ -291,10 +306,25 @@
# We don't want to be fooled by proxies.
klass = type(obj)
+ # We want to serialize persistent classes by name if they have
+ # a non-None non-empty module so as not to have a direct
+ # ref. This is important when copying. We probably want to
+ # revisit this in the future.
newargs = getattr(obj, "__getnewargs__", None)
- if newargs is None:
+ if (isinstance(getattr(klass, '_p_oid', 0), _oidtypes)
+ and klass.__module__):
+ # This is a persistent class with a non-empty module. This
+ # uses pickle format #3 or #7.
+ klass = klass.__module__, klass.__name__
+ if newargs is None:
+ meta = klass, None
+ else:
+ meta = klass, newargs()
+ elif newargs is None:
+ # Pickle format #1.
meta = klass
else:
+ # Pickle format #2.
meta = klass, newargs()
return self._dump(meta, obj.__getstate__())
Modified: ZODB/branches/3.4/src/ZODB/tests/sampledm.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/sampledm.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/sampledm.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -405,7 +405,7 @@
def test_suite():
- from doctest import DocTestSuite
+ from zope.testing.doctest import DocTestSuite
return DocTestSuite()
if __name__ == '__main__':
Modified: ZODB/branches/3.4/src/ZODB/tests/testBroken.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/testBroken.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/testBroken.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -20,7 +20,7 @@
import unittest
import persistent
import transaction
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
from ZODB.tests.util import DB
def test_integration():
Modified: ZODB/branches/3.4/src/ZODB/tests/testConnection.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/testConnection.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/testConnection.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -13,7 +13,7 @@
##############################################################################
"""Unit tests for the Connection class."""
-import doctest
+from zope.testing import doctest
import unittest
import warnings
Modified: ZODB/branches/3.4/src/ZODB/tests/testFileStorage.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/testFileStorage.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/testFileStorage.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -485,7 +485,7 @@
"""
def test_suite():
- import doctest
+ from zope.testing import doctest
suite = unittest.TestSuite()
for klass in [FileStorageTests, Corruption.FileStorageCorruptTests,
Modified: ZODB/branches/3.4/src/ZODB/tests/testSerialize.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/testSerialize.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/testSerialize.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -121,7 +121,7 @@
def test_suite():
- import doctest
+ from zope.testing import doctest
suite = unittest.makeSuite(SerializerTestCase)
suite.addTest(doctest.DocTestSuite("ZODB.serialize"))
return suite
Modified: ZODB/branches/3.4/src/ZODB/tests/testSubTransaction.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/testSubTransaction.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/testSubTransaction.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -132,7 +132,7 @@
"""
-import doctest
+from zope.testing import doctest
def test_suite():
return doctest.DocTestSuite()
Modified: ZODB/branches/3.4/src/ZODB/tests/test_cache.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/test_cache.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/test_cache.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -13,7 +13,7 @@
##############################################################################
"""Test behavior of Connection plus cPickleCache."""
-import doctest
+from zope.testing import doctest
from persistent import Persistent
import transaction
Modified: ZODB/branches/3.4/src/ZODB/tests/test_datamanageradapter.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/test_datamanageradapter.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/test_datamanageradapter.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -15,7 +15,7 @@
$Id$
"""
import unittest
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
from transaction._transaction import DataManagerAdapter
from ZODB.tests.sampledm import DataManager
Modified: ZODB/branches/3.4/src/ZODB/tests/testmvcc.py
===================================================================
--- ZODB/branches/3.4/src/ZODB/tests/testmvcc.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/testmvcc.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -358,7 +358,7 @@
"""
-import doctest
+from zope.testing import doctest
def test_suite():
return doctest.DocTestSuite()
Copied: ZODB/branches/3.4/src/ZODB/tests/testpersistentclass.py (from rev 29890, Zope/branches/jim-fix-zclasses/lib/python/ZODB/tests/testpersistentclass.py)
===================================================================
--- Zope/branches/jim-fix-zclasses/lib/python/ZODB/tests/testpersistentclass.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/ZODB/tests/testpersistentclass.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -0,0 +1,51 @@
+##############################################################################
+#
+# Copyright (c) 2004 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""ZClass tests
+
+$Id$
+"""
+
+import os, sys
+import unittest
+import ZODB.tests.util
+import transaction
+from zope.testing import doctest
+
+
+# XXX need to update files to get newer testing package
+class FakeModule:
+ def __init__(self, name, dict):
+ self.__dict__ = dict
+ self.__name__ = name
+
+
+def setUp(test):
+ test.globs['some_database'] = ZODB.tests.util.DB()
+ module = FakeModule('ZODB.persistentclass_txt', test.globs)
+ sys.modules[module.__name__] = module
+
+def tearDown(test):
+ transaction.abort()
+ test.globs['some_database'].close()
+ del sys.modules['ZODB.persistentclass_txt']
+
+def test_suite():
+ return unittest.TestSuite((
+ doctest.DocFileSuite("../persistentclass.txt",
+ setUp=setUp, tearDown=tearDown),
+ ))
+
+if __name__ == '__main__':
+ unittest.main(defaultTest='test_suite')
+
Property changes on: ZODB/branches/3.4/src/ZODB/tests/testpersistentclass.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Modified: ZODB/branches/3.4/src/persistent/cPickleCache.c
===================================================================
--- ZODB/branches/3.4/src/persistent/cPickleCache.c 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/persistent/cPickleCache.c 2005-04-06 22:35:05 UTC (rev 29891)
@@ -115,7 +115,6 @@
int klass_count; /* count of persistent classes */
PyObject *data; /* oid -> object dict */
PyObject *jar; /* Connection object */
- PyObject *setklassstate; /* ??? */
int cache_size; /* target number of items in cache */
/* Most of the time the ring contains only:
@@ -331,58 +330,48 @@
return lockgc(self, 0);
}
-static void
+static int
_invalidate(ccobject *self, PyObject *key)
{
- static PyObject *_p_invalidate;
- PyObject *v = PyDict_GetItem(self->data, key);
+ static PyObject *_p_invalidate = NULL;
+ PyObject *meth, *v;
- if (!_p_invalidate) {
+ v = PyDict_GetItem(self->data, key);
+ if (v == NULL)
+ return 0;
+
+ if (_p_invalidate == NULL)
+ {
_p_invalidate = PyString_InternFromString("_p_invalidate");
- if (!_p_invalidate) {
+ if (_p_invalidate == NULL)
+ {
/* It doesn't make any sense to ignore this error, but
the caller ignores all errors.
+
+ TODO: and why does it do that? This should be fixed
*/
- PyErr_Clear();
- return;
- }
- }
+ return -1;
+ }
+ }
- if (!v)
- return;
- if (PyType_Check(v)) {
- /* This looks wrong, but it isn't. We use strong references to types
- because they don't have the ring members.
+ if (v->ob_refcnt <= 1 && PyType_Check(v)) {
+ /* This looks wrong, but it isn't. We use strong references to types
+ because they don't have the ring members.
- The result is that we *never* remove classes unless
- they are modified.
+ The result is that we *never* remove classes unless
+ they are modified. We can fix this by using wekrefs uniformly.
+ */
+ self->klass_count--;
+ return PyDict_DelItem(self->data, key);
+ }
- */
- if (v->ob_refcnt <= 1) {
- self->klass_count--;
- if (PyDict_DelItem(self->data, key) < 0)
- PyErr_Clear();
- }
- else {
- v = PyObject_CallFunction(self->setklassstate, "O", v);
- if (v)
- Py_DECREF(v);
- else
- PyErr_Clear();
- }
- } else {
- PyObject *meth, *err;
+ meth = PyObject_GetAttr(v, _p_invalidate);
+ if (meth == NULL)
+ return -1;
- meth = PyObject_GetAttr(v, _p_invalidate);
- if (!meth) {
- PyErr_Clear();
- return;
- }
- err = PyObject_CallObject(meth, NULL);
- Py_DECREF(meth);
- if (!err)
- PyErr_Clear();
- }
+ v = PyObject_CallObject(meth, NULL);
+ Py_DECREF(meth);
+ return v == NULL ? -1 : 0;
}
static PyObject *
@@ -391,16 +380,23 @@
PyObject *key, *v;
int i = 0;
- if (PyDict_Check(inv)) {
+ if (PyDict_Check(inv))
+ {
while (PyDict_Next(inv, &i, &key, &v))
- _invalidate(self, key);
+ {
+ if (_invalidate(self, key) < 0)
+ return NULL;
+ }
PyDict_Clear(inv);
- }
+ }
else {
if (PyString_Check(inv))
- _invalidate(self, inv);
+ {
+ if (_invalidate(self, inv) < 0)
+ return NULL;
+ }
else {
- int l;
+ int l, r;
l = PyObject_Length(inv);
if (l < 0)
@@ -409,8 +405,10 @@
key = PySequence_GetItem(inv, i);
if (!key)
return NULL;
- _invalidate(self, key);
+ r = _invalidate(self, key);
Py_DECREF(key);
+ if (r < 0)
+ return NULL;
}
/* Dubious: modifying the input may be an unexpected side effect. */
PySequence_DelSlice(inv, 0, l);
@@ -669,7 +667,7 @@
if (!PyArg_ParseTuple(args, "O|i", &jar, &cache_size))
return -1;
- self->setklassstate = self->jar = NULL;
+ self->jar = NULL;
self->data = PyDict_New();
if (self->data == NULL) {
Py_DECREF(self);
@@ -686,11 +684,6 @@
non-ghost objects.
*/
PyObject_GC_UnTrack((void *)self->data);
- self->setklassstate = PyObject_GetAttrString(jar, "setklassstate");
- if (self->setklassstate == NULL) {
- Py_DECREF(self);
- return -1;
- }
self->jar = jar;
Py_INCREF(jar);
self->cache_size = cache_size;
@@ -708,7 +701,6 @@
{
Py_XDECREF(self->data);
Py_XDECREF(self->jar);
- Py_XDECREF(self->setklassstate);
PyObject_GC_Del(self);
}
@@ -755,7 +747,6 @@
}
Py_XDECREF(self->jar);
- Py_XDECREF(self->setklassstate);
while (PyDict_Next(self->data, &pos, &k, &v)) {
Py_INCREF(v);
@@ -765,7 +756,6 @@
Py_XDECREF(self->data);
self->data = NULL;
self->jar = NULL;
- self->setklassstate = NULL;
return 0;
}
@@ -794,7 +784,6 @@
}
VISIT(self->jar);
- VISIT(self->setklassstate);
here = self->ring_home.r_next;
Modified: ZODB/branches/3.4/src/persistent/tests/test_PickleCache.py
===================================================================
--- ZODB/branches/3.4/src/persistent/tests/test_PickleCache.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/persistent/tests/test_PickleCache.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -40,7 +40,7 @@
"""
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
import unittest
def test_suite():
Modified: ZODB/branches/3.4/src/persistent/tests/test_overriding_attrs.py
===================================================================
--- ZODB/branches/3.4/src/persistent/tests/test_overriding_attrs.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/persistent/tests/test_overriding_attrs.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -398,5 +398,5 @@
def test_suite():
- from doctest import DocTestSuite
+ from zope.testing.doctest import DocTestSuite
return DocTestSuite()
Modified: ZODB/branches/3.4/src/persistent/tests/test_persistent.py
===================================================================
--- ZODB/branches/3.4/src/persistent/tests/test_persistent.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/persistent/tests/test_persistent.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -11,11 +11,9 @@
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
-
+from zope.testing import doctest
from persistent import Persistent
-from zope.testing.doctestunit import DocFileSuite
-
class P(Persistent):
def __init__(self):
self.x = 0
@@ -23,4 +21,4 @@
self.x += 1
def test_suite():
- return DocFileSuite("persistent.txt", globs={"P": P})
+ return doctest.DocFileSuite("persistent.txt", globs={"P": P})
Modified: ZODB/branches/3.4/src/persistent/tests/test_pickle.py
===================================================================
--- ZODB/branches/3.4/src/persistent/tests/test_pickle.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/persistent/tests/test_pickle.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -269,7 +269,7 @@
"""
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
import unittest
def test_suite():
Modified: ZODB/branches/3.4/src/persistent/tests/test_wref.py
===================================================================
--- ZODB/branches/3.4/src/persistent/tests/test_wref.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/persistent/tests/test_wref.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -15,7 +15,7 @@
$Id$
"""
import unittest
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
def test_suite():
return DocTestSuite('persistent.wref')
Modified: ZODB/branches/3.4/src/transaction/__init__.py
===================================================================
--- ZODB/branches/3.4/src/transaction/__init__.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/transaction/__init__.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -20,19 +20,11 @@
from transaction._manager import TransactionManager, ThreadTransactionManager
manager = ThreadTransactionManager()
+get = manager.get
+begin = manager.begin
+commit = manager.commit
+abort = manager.abort
-def get():
- return manager.get()
-
-def begin():
- return manager.begin()
-
-def commit(sub=False):
- manager.get().commit(sub)
-
-def abort(sub=False):
- manager.get().abort(sub)
-
def get_transaction():
from ZODB.utils import deprecated36
deprecated36(""" use transaction.get() instead of get_transaction().
Modified: ZODB/branches/3.4/src/transaction/_manager.py
===================================================================
--- ZODB/branches/3.4/src/transaction/_manager.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/transaction/_manager.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -61,7 +61,13 @@
def unregisterSynch(self, synch):
self._synchs.remove(synch)
-class ThreadTransactionManager(object):
+ def commit(self, sub=False):
+ self.get().commit(sub)
+
+ def abort(self, sub=False):
+ self.get().abort(sub)
+
+class ThreadTransactionManager(TransactionManager):
"""Thread-aware transaction manager.
Each thread is associated with a unique transaction.
Modified: ZODB/branches/3.4/src/transaction/_transaction.py
===================================================================
--- ZODB/branches/3.4/src/transaction/_transaction.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/transaction/_transaction.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -54,8 +54,10 @@
The second argument to tpc_begin() indicates that a subtransaction
commit is beginning (if it is true). In a subtransaction, there is no
-tpc_vote() call (I don't know why not). The tpc_finish()
-or tpc_abort() call applies just to that subtransaction.
+tpc_vote() call, because sub-transactions don't need 2-phase commit.
+If a sub-transaction abort or commit fails, we can abort the outer
+transaction. The tpc_finish() or tpc_abort() call applies just to
+that subtransaction.
Once a resource manager is involved in a subtransaction, all
subsequent transactions will be treated as subtransactions until
@@ -244,6 +246,8 @@
# commit protocol.
manager = getattr(obj, "_p_jar", obj)
+ if manager is None:
+ raise ValueError("Register with no manager")
adapter = self._adapters.get(manager)
if adapter is None:
if myhasattr(manager, "commit_sub"):
Modified: ZODB/branches/3.4/src/transaction/tests/test_SampleDataManager.py
===================================================================
--- ZODB/branches/3.4/src/transaction/tests/test_SampleDataManager.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/transaction/tests/test_SampleDataManager.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -405,7 +405,7 @@
def test_suite():
- from doctest import DocTestSuite
+ from zope.testing.doctest import DocTestSuite
return DocTestSuite()
if __name__ == '__main__':
Modified: ZODB/branches/3.4/src/transaction/tests/test_register_compat.py
===================================================================
--- ZODB/branches/3.4/src/transaction/tests/test_register_compat.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/transaction/tests/test_register_compat.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -148,7 +148,7 @@
obj.abort()
self.aborted.append(obj)
-import doctest
+from zope.testing import doctest
def test_suite():
return doctest.DocTestSuite()
Modified: ZODB/branches/3.4/src/transaction/tests/test_transaction.py
===================================================================
--- ZODB/branches/3.4/src/transaction/tests/test_transaction.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/transaction/tests/test_transaction.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -46,16 +46,12 @@
class TransactionTests(unittest.TestCase):
def setUp(self):
- self.orig_tm = transaction.manager
- transaction.manager = transaction.TransactionManager()
- self.sub1 = DataObject()
- self.sub2 = DataObject()
- self.sub3 = DataObject()
- self.nosub1 = DataObject(nost=1)
+ self.txn_mgr = transaction.TransactionManager()
+ self.sub1 = DataObject(self.txn_mgr)
+ self.sub2 = DataObject(self.txn_mgr)
+ self.sub3 = DataObject(self.txn_mgr)
+ self.nosub1 = DataObject(self.txn_mgr, nost=1)
- def tearDown(self):
- transaction.manager = self.orig_tm
-
# basic tests with two sub trans jars
# really we only need one, so tests for
# sub1 should identical to tests for sub2
@@ -64,7 +60,7 @@
self.sub1.modify()
self.sub2.modify()
- transaction.commit()
+ self.txn_mgr.commit()
assert self.sub1._p_jar.ccommit_sub == 0
assert self.sub1._p_jar.ctpc_finish == 1
@@ -74,13 +70,13 @@
self.sub1.modify()
self.sub2.modify()
- transaction.abort()
+ self.txn_mgr.abort()
assert self.sub2._p_jar.cabort == 1
def testTransactionNote(self):
- t = transaction.get()
+ t = self.txn_mgr.get()
t.note('This is a note.')
self.assertEqual(t.description, 'This is a note.')
@@ -94,12 +90,12 @@
self.sub1.modify()
self.sub2.modify()
- transaction.commit(1)
+ self.txn_mgr.commit(1)
assert self.sub1._p_jar.ctpc_vote == 0
assert self.sub1._p_jar.ctpc_finish == 1
- transaction.commit()
+ self.txn_mgr.commit()
assert self.sub1._p_jar.ccommit_sub == 1
assert self.sub1._p_jar.ctpc_vote == 1
@@ -109,8 +105,8 @@
self.sub1.modify()
self.sub2.modify()
- transaction.commit(1)
- transaction.abort()
+ self.txn_mgr.commit(1)
+ self.txn_mgr.abort()
assert self.sub1._p_jar.ctpc_vote == 0
assert self.sub1._p_jar.cabort == 0
@@ -118,12 +114,12 @@
def testMultipleSubTransactionCommitCommit(self):
self.sub1.modify()
- transaction.commit(1)
+ self.txn_mgr.commit(1)
self.sub2.modify()
# reset a flag on the original to test it again
self.sub1.ctpc_finish = 0
- transaction.commit(1)
+ self.txn_mgr.commit(1)
# this is interesting.. we go through
# every subtrans commit with all subtrans capable
@@ -135,7 +131,7 @@
# add another before we do the entire txn commit
self.sub3.modify()
- transaction.commit()
+ self.txn_mgr.commit()
# we did an implicit sub commit, is this impl artifact?
assert self.sub3._p_jar.ccommit_sub == 1
@@ -161,12 +157,12 @@
# add it
self.sub1.modify()
- transaction.commit(1)
+ self.txn_mgr.commit(1)
# add another
self.sub2.modify()
- transaction.commit(1)
+ self.txn_mgr.commit(1)
assert self.sub1._p_jar.ctpc_vote == 0
assert self.sub1._p_jar.ctpc_finish > 0
@@ -175,10 +171,10 @@
self.sub3.modify()
# abort the sub transaction
- transaction.abort(1)
+ self.txn_mgr.abort(1)
# commit the container transaction
- transaction.commit()
+ self.txn_mgr.commit()
assert self.sub3._p_jar.cabort == 1
assert self.sub1._p_jar.ccommit_sub == 1
@@ -190,7 +186,7 @@
self.nosub1.modify()
- transaction.commit()
+ self.txn_mgr.commit()
assert self.nosub1._p_jar.ctpc_finish == 1
@@ -198,7 +194,7 @@
self.nosub1.modify()
- transaction.abort()
+ self.txn_mgr.abort()
assert self.nosub1._p_jar.ctpc_finish == 0
assert self.nosub1._p_jar.cabort == 1
@@ -221,7 +217,7 @@
self.sub1.modify(tracing='sub')
self.nosub1.modify(tracing='nosub')
- transaction.commit(1)
+ self.txn_mgr.commit(1)
assert self.sub1._p_jar.ctpc_finish == 1
@@ -229,7 +225,7 @@
# in a subtrans
assert self.nosub1._p_jar.ctpc_finish == 0
- transaction.abort()
+ self.txn_mgr.abort()
assert self.nosub1._p_jar.cabort == 1
assert self.sub1._p_jar.cabort_sub == 1
@@ -239,11 +235,11 @@
self.sub1.modify()
self.nosub1.modify()
- transaction.commit(1)
+ self.txn_mgr.commit(1)
assert self.nosub1._p_jar.ctpc_vote == 0
- transaction.commit()
+ self.txn_mgr.commit()
#assert self.nosub1._p_jar.ccommit_sub == 0
assert self.nosub1._p_jar.ctpc_vote == 1
@@ -276,12 +272,12 @@
# add it
self.sub1.modify()
- transaction.commit(1)
+ self.txn_mgr.commit(1)
# add another
self.nosub1.modify()
- transaction.commit(1)
+ self.txn_mgr.commit(1)
assert self.sub1._p_jar.ctpc_vote == 0
assert self.nosub1._p_jar.ctpc_vote == 0
@@ -291,7 +287,7 @@
self.sub2.modify()
# commit the container transaction
- transaction.commit()
+ self.txn_mgr.commit()
# we did an implicit sub commit
assert self.sub2._p_jar.ccommit_sub == 1
@@ -316,7 +312,7 @@
self.sub2.modify()
try:
- transaction.abort()
+ self.txn_mgr.abort()
except TestTxnException: pass
assert self.nosub1._p_jar.cabort == 1
@@ -330,7 +326,7 @@
self.sub1.modify(nojar=1)
try:
- transaction.commit()
+ self.txn_mgr.commit()
except TestTxnException: pass
assert self.nosub1._p_jar.ctpc_finish == 0
@@ -345,7 +341,7 @@
self.sub1.modify(nojar=1)
try:
- transaction.commit()
+ self.txn_mgr.commit()
except TestTxnException: pass
assert self.nosub1._p_jar.ctpc_finish == 0
@@ -371,7 +367,7 @@
self.sub1.modify(nojar=1)
try:
- transaction.commit()
+ self.txn_mgr.commit()
except TestTxnException: pass
assert self.nosub1._p_jar.ctpc_abort == 1
@@ -385,7 +381,7 @@
self.sub1.modify(nojar=1)
try:
- transaction.commit()
+ self.txn_mgr.commit()
except TestTxnException:
pass
@@ -402,19 +398,19 @@
# they come out of the dictionary.
self.sub1.modify()
- transaction.commit(1)
+ self.txn_mgr.commit(1)
self.nosub1.modify()
self.sub2._p_jar = SubTransactionJar(errors='commit_sub')
self.sub2.modify(nojar=1)
- transaction.commit(1)
+ self.txn_mgr.commit(1)
self.sub3.modify()
try:
- transaction.commit()
+ self.txn_mgr.commit()
except TestTxnException:
pass
@@ -441,17 +437,17 @@
self.sub1._p_jar = SubTransactionJar(errors='commit_sub')
self.sub1.modify(nojar=1)
- transaction.commit(1)
+ self.txn_mgr.commit(1)
self.nosub1.modify()
self.sub2._p_jar = SubTransactionJar(errors='abort_sub')
self.sub2.modify(nojar=1)
- transaction.commit(1)
+ self.txn_mgr.commit(1)
self.sub3.modify()
try:
- transaction.commit()
+ self.txn_mgr.commit()
except TestTxnException, err:
pass
else:
@@ -500,7 +496,8 @@
class DataObject:
- def __init__(self, nost=0):
+ def __init__(self, txn_mgr, nost=0):
+ self.txn_mgr = txn_mgr
self.nost = nost
self._p_jar = None
@@ -510,7 +507,7 @@
self._p_jar = NoSubTransactionJar(tracing=tracing)
else:
self._p_jar = SubTransactionJar(tracing=tracing)
- transaction.get().register(self)
+ self.txn_mgr.get().register(self)
class TestTxnException(Exception):
pass
@@ -635,7 +632,7 @@
"""
def test_suite():
- from doctest import DocTestSuite
+ from zope.testing.doctest import DocTestSuite
return unittest.TestSuite((
DocTestSuite(),
unittest.makeSuite(TransactionTests),
Modified: ZODB/branches/3.4/src/transaction/tests/test_util.py
===================================================================
--- ZODB/branches/3.4/src/transaction/tests/test_util.py 2005-04-06 15:57:54 UTC (rev 29890)
+++ ZODB/branches/3.4/src/transaction/tests/test_util.py 2005-04-06 22:35:05 UTC (rev 29891)
@@ -16,7 +16,7 @@
$Id$
"""
import unittest
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
def test_suite():
return DocTestSuite('transaction.util')
More information about the Zodb-checkins
mailing list