[ZODB-Dev] nontransactional, oid, and thread safety. oh my!
Randy
randito at gmail.com
Mon Oct 4 11:51:58 EDT 2004
Thanks for all of the great information!
Here is the approach I decided to go with.
I am going to use the FileStorage storage, mechanism. And, instead of
commiting every change to the database, I am going to do a bunch of
related changes, and them commit them all at once. (According to the
situation, of course). This has reduced the size of my database, and
the need to pack as often.
I tried the Berkley storage, but it didn't perform as expected.
Second, as previously pointed out, the Berkley storage is no longer
supported. Third, there is no number 3.
Here are the sample Database and Connection abstractions that I'm
using. The intent is that the Database objects (which are cached in
my code at a higher level) will hand off Connection objects for
incoming (XMLRPC in case you are wondering) requests.
Please don't laugh out loud when you see my code. I'm still learing.
class Connection:
"""
A database connection class used to access database data.
A persistant dictionary is present at Connection.data that can be
used to write data.
Keys into the dictionary should be strings.
Use commit() to save your changes, and abort() to rollback your changes.
CreateId() will create a unique id for you.
When finished, use Data.close() to close the connection. Very important!
"""
def __init__(self, database):
assert isinstance(database, Database)
global DATAKEY
# create a transient connection, that will be closed when this
object is finished
self.database = database
self.conn = self.database.db.open()
root = self.conn.root()
self.data = root[DATAKEY]
def __del__(self):
self.close()
def __repr__(self):
return "Connection %s" % self.database.path
def commit(self):
get_transaction().commit()
def abort(self):
get_transaction().abort()
def createId(self):
# todo: this is a potential database "hotspot".. look into alternatives
global IDKEY
root = self.conn.root()
assert root.has_key(IDKEY)
id = long(root[IDKEY])
id = id + 1L
root[IDKEY] = str(id) # save ID as string. keys in database are
always strings
self.commit()
return id
def close(self):
if self.conn:
print "Closing database connection to %s" % self.database.path
self.conn.close()
self.conn = None
self.data = None
class Database:
def __init__(self, path):
self.path = path
self.storage = ZODB.FileStorage.FileStorage(path)
self.db = ZODB.DB(self.storage)
self.packing = False
self._initialize()
def __repr__(self):
return "Database %s" % self.path
def _initialize(self):
global DATAKEY, IDKEY
c = self.db.open()
root = c.root()
# high level data table is a btree optimized with string
(object) keys and object data
if not root.has_key(DATAKEY):
root[DATAKEY] = BTrees.OOBTree.OOBTree()
if not root.has_key(IDKEY):
root[IDKEY] = str(0L)
get_transaction().commit()
c.close()
def connection(self):
"""Get a connection type data object that can be used to write
to the database"""
return Connection(self)
def close(self):
if self.db and self.storage:
print "Closing database %s" % self.path
self.db.close()
self.storage.close()
self.db = None
self.storage = None
def __del__(self):
self.close()
> [Jim Fulton]
> > Right, but, there are storage implementations that don't require packing.
> >
> > In particular, the Berkeley DB storage doesn't need to be packed at all
> > of you avoid cyclic references among persistent objects and doesn't need
> > to be packed to get rid of old revisions. (Actually, it packs
> > automatically.) With this storage, you get the benefits of transactions
> > without getting large database growth. You *do* then need to follow best
> > Berkeley DB practices, like log-file management.
>
> Toby mentioned BDBStorage too, but I deliberately didn't: BDB isn't
> supported in the ZODB 3.2 line anymore, and the code for it doesn't even
> exist in ZODB 3.3. If Randy wants to rehabilitate and take over that code,
> great, but I assumed he was asking for an approach he could "just use".
>
>
More information about the ZODB-Dev
mailing list