[Checkins] SVN: mongopersist/trunk/ - Performance: Drastically improved performance for collections that store
Stephen Richter
cvs-admin at zope.org
Sun Apr 1 01:27:24 UTC 2012
Log message for revision 124834:
- Performance: Drastically improved performance for collections that store
only one type of objects and where the documents do not store the type
(i.e. it is stored in the name map collection).
Changed:
U mongopersist/trunk/CHANGES.txt
U mongopersist/trunk/src/mongopersist/serialize.py
-=-
Modified: mongopersist/trunk/CHANGES.txt
===================================================================
--- mongopersist/trunk/CHANGES.txt 2012-03-31 20:29:37 UTC (rev 124833)
+++ mongopersist/trunk/CHANGES.txt 2012-04-01 01:27:20 UTC (rev 124834)
@@ -47,6 +47,10 @@
- Feature: Added transaction ID to LoggingDecorator.
+- Performance: Drastically improved performance for collections that store
+ only one type of objects and where the documents do not store the type
+ (i.e. it is stored in the name map collection).
+
- Bug: We have seen several occasions in production where we suddenly lost
some state in some documents, which prohibited the objects from being
loadable again. The cause was that the ``_original_states`` attribute did not
Modified: mongopersist/trunk/src/mongopersist/serialize.py
===================================================================
--- mongopersist/trunk/src/mongopersist/serialize.py 2012-03-31 20:29:37 UTC (rev 124833)
+++ mongopersist/trunk/src/mongopersist/serialize.py 2012-04-01 01:27:20 UTC (rev 124834)
@@ -278,33 +278,46 @@
def __init__(self, jar):
self._jar = jar
+ self._single_map_cache = {}
def simple_resolve(self, path):
return resolve(path)
def resolve(self, dbref):
__traceback_info__ = dbref
+ # 1. Check the global oid-based lookup cache.
try:
return OID_CLASS_LRU[dbref.id]
except KeyError:
pass
- # First we try to resolve the path directly.
+ # 2. Check the transient single map entry lookup cache.
try:
+ return self._single_map_cache[(dbref.database, dbref.collection)]
+ except KeyError:
+ pass
+ # 3. Try to resolve the path directly.
+ try:
return self.simple_resolve(dbref.collection)
except ImportError:
pass
+ # 4. No simple hits, so we have to do some leg work.
# Let's now try to look up the path from the collection to path
# mapping
db = self._jar._conn[self._jar.default_database]
coll = db[self._jar.name_map_collection]
result = coll.find(
{'collection': dbref.collection, 'database': dbref.database})
- if result.count() == 0:
+ count = result.count()
+ if count == 0:
raise ImportError(dbref)
- elif result.count() == 1:
+ elif count == 1:
# Do not add these results to the LRU cache, since the count might
- # change later.
- return self.simple_resolve(result.next()['path'])
+ # change later. But storing it for the length of the transaction
+ # is fine, which is really useful if you load a lot of objects of
+ # the same type.
+ klass = self.simple_resolve(result.next()['path'])
+ self._single_map_cache[(dbref.database, dbref.collection)] = klass
+ return klass
else:
if dbref.id is None:
raise ImportError(dbref)
More information about the checkins
mailing list