[Zope-CVS] CVS: Products/AdaptableStorage - CHANGES.txt:1.3.2.1 README.txt:1.3.2.1 dbtab.conf:1.1.2.1 TmpStore.py:1.2.2.1 Zope2FS.py:1.16.2.2 Zope2Mapper.py:1.3.2.2 Zope2SQL.py:1.7.2.2 doc.txt:1.4.2.1 patches.py:1.2.2.1 version.txt:1.4.2.1
Christian Zagrodnick
cz@gocept.com
Tue, 21 Jan 2003 03:11:46 -0500
Update of /cvs-repository/Products/AdaptableStorage
In directory cvs.zope.org:/tmp/cvs-serv25960
Modified Files:
Tag: zagy-patches
TmpStore.py Zope2FS.py Zope2Mapper.py Zope2SQL.py doc.txt
patches.py version.txt
Added Files:
Tag: zagy-patches
CHANGES.txt README.txt dbtab.conf
Log Message:
merging HEAD into zagy-patches branch
=== Added File Products/AdaptableStorage/CHANGES.txt ===
Version 0.4.2
- Corrected serialization of large OFS.File objects. OFS.File uses
some ZODB trickery to store large files, but AdaptableStorage needs
to be aware of that trickery.
- Made the remainder pickler properly restore cyclic references to
the persistent object. Until now, it accidentally made internal
copies of the object.
- Although it involved no changes to AdaptableStorage, fixed some
bugs in ZCatalog that prevented them from being stored by
AdaptableStorage.
- Fixed object copying. The object copier assumed it could ghostify
objects immediately, but it turns out that the pickling machinery
sometimes needs to revisit objects.
Version 0.4.1
First public release.
=== Added File Products/AdaptableStorage/README.txt ===
* AdaptableStorage Quick Start *
The following steps help you get started with AdaptableStorage quickly.
See doc.txt for more information.
1. Check your Zope version. You need at least Zope 2.6.0, but a later
version is recommended.
2. Add the AdaptableStorage product to Zope by placing it in the
Products directory. If you're using an INSTANCE_HOME setup, place it
with your other add-on products.
3. Install version 1.0.2 or later of the DBTab product. After adding
the product to Zope, set up custom_zodb.py. If you're using Unix,
make a symlink:
cd <zope_root>
ln -s lib/python/Products/DBTab/custom_zodb.py
If you're using Windows, make a copy instead. If you're using an
INSTANCE_HOME setup, adjust the path to custom_zodb.py accordingly.
4. Copy the sample dbtab.conf from the AdaptableStorage product to
your INSTANCE_HOME or the root of your Zope site.
5. Start Zope. If Zope refuses to start because "no root database is
configured", make sure dbtab.conf is in the right place and try again.
6. Visit the Zope management interface. Select "DBTab Mount Point"
from the "add" drop-down. The checkbox next to "/fs" should be
enabled. Click the "Create selected mount points" button.
7. Visit the new folder /fs and add things to it. Objects added to
/fs get stored in the filesystem directory <zope>/var/mnt/fs. You can
also add files to <zope>/var/mnt/fs and see them appear in Zope.
8. To speed up access to /fs, enable the ZODB cache by setting the
cache size in dbtab.conf to 4000 objects. The sample dbtab.conf sets
it to 0, which effectively clears the cache after each request to
Zope. With the cache enabled, access to /fs is just as fast as access
to other Zope objects, but changes made on the filesystem won't be
reflected immediately in Zope.
=== Added File Products/AdaptableStorage/dbtab.conf ===
# A sample dbtab.conf for getting started with AdaptableStorage.
[Storage: Main]
type=FileStorage
file_name=%(CLIENT_HOME)s/Data.fs
[Database: Main]
mount_paths=/
[Storage: FS]
type=ASStorage
factory=Products.AdaptableStorage.Zope2FS.createMapper
basepath=%(CLIENT_HOME)s/mnt
[Database: FS]
class=ASDB
cache_size=0
mount_paths=/fs
auto_create=1
=== Products/AdaptableStorage/TmpStore.py 1.2 => 1.2.2.1 ===
--- Products/AdaptableStorage/TmpStore.py:1.2 Tue Dec 31 16:15:22 2002
+++ Products/AdaptableStorage/TmpStore.py Tue Jan 21 03:11:11 2003
@@ -13,7 +13,7 @@
##############################################################################
from ZODB import POSException
-from ZODB.utils import p64, u64, z64
+from ZODB.utils import p64, u64
import tempfile
@@ -85,7 +85,7 @@
self._file.seek(self._pos)
l = len(data)
if serial is None:
- serial = z64
+ serial = '\0' * 8
header = p64(len(oid)) + oid + serial + p64(l)
self._file.write(header)
self._file.write(data)
=== Products/AdaptableStorage/Zope2FS.py 1.16.2.1 => 1.16.2.2 ===
--- Products/AdaptableStorage/Zope2FS.py:1.16.2.1 Sat Jan 4 14:09:18 2003
+++ Products/AdaptableStorage/Zope2FS.py Tue Jan 21 03:11:11 2003
@@ -20,7 +20,8 @@
from mapper.public import ObjectGateway
from mapper_std.public import PathKeychainGenerator
from gateway_fs.public import FSConnection, FSDirectoryItems, FSAutoId, \
- FSSectionData, FSClassificationSection, FSFileData, FSProperties
+ FSSectionData, FSClassificationSection, FSFileData, FSProperties, \
+ FSUserList
from Zope2Mapper import createZope2Mapper
@@ -55,6 +56,36 @@
g.addGateway('remainder', FSSectionData(conn, 'remainder'))
root_mapper.getSubMapper('ZopePageTemplate').setGateway(g)
+ # dtml method gateway
+ g = ObjectGateway()
+ g.addGateway('text', FSFileData(conn))
+ g.addGateway('id', FSAutoId())
+ g.addGateway('remainder', FSSectionData(conn, 'remainder'))
+ root_mapper.getSubMapper('OFS.DTMLMethod.DTMLMethod').setGateway(g)
+
+ # dtml document gateway
+ g = ObjectGateway()
+ g.addGateway('text', FSFileData(conn))
+ g.addGateway('properties', properties_gw)
+ g.addGateway('id', FSAutoId())
+ g.addGateway('remainder', FSSectionData(conn, 'remainder'))
+ root_mapper.getSubMapper('OFS.DTMLDocument.DTMLDocument').setGateway(g)
+
+ # zsql mapper
+ g = ObjectGateway()
+ g.addGateway('text', FSFileData(conn))
+ g.addGateway('properties', FSProperties(conn, 'ZSQL Properties'))
+ g.addGateway('id', FSAutoId())
+ g.addGateway('remainder', FSSectionData(conn, 'remainder'))
+ root_mapper.getSubMapper('Products.ZSQLMethods.SQL.SQL').setGateway(g)
+
+ # python script mapper
+ g = ObjectGateway()
+ g.addGateway('body', FSFileData(conn))
+ g.addGateway('id', FSAutoId())
+ g.addGateway('remainder', FSSectionData(conn, 'remainder'))
+ root_mapper.getSubMapper('PythonScript').setGateway(g)
+
# file gateway
g = ObjectGateway()
g.addGateway('data', FSFileData(conn))
@@ -66,10 +97,17 @@
# image gateway is identical
root_mapper.getSubMapper('OFS.Image.Image').setGateway(g)
+ # user folder gateway
+ g = ObjectGateway()
+ g.addGateway('data', FSUserList(conn))
+ g.addGateway('id', FSAutoId())
+ g.addGateway('remainder', FSSectionData(conn, 'remainder'))
+ root_mapper.getSubMapper('AccessControl.User.UserFolder').setGateway(g)
+
# anyfolder object gateway
g = ObjectGateway()
g.addGateway('items', FSDirectoryItems(conn))
- #g.addGateway('properties', properties_gw)
+ g.addGateway('properties', properties_gw)
g.addGateway('id', FSAutoId())
g.addGateway('remainder', FSSectionData(conn, 'remainder'))
root_mapper.getSubMapper('anyfolder').setGateway(g)
=== Products/AdaptableStorage/Zope2Mapper.py 1.3.2.1 => 1.3.2.2 ===
--- Products/AdaptableStorage/Zope2Mapper.py:1.3.2.1 Sat Jan 4 14:09:18 2003
+++ Products/AdaptableStorage/Zope2Mapper.py Tue Jan 21 03:11:11 2003
@@ -21,10 +21,12 @@
import ObjectSerializer, ObjectGateway, ObjectMapper
from mapper_std.public \
import RollCall, FixedPersistentMapping, IgnoredAttribute, \
- AnyObjectSerializer, StringDataAttribute
+ AnyObjectSerializer, StringDataAttribute, OptionalAspect
from serial_ofs.public \
import FolderItems, FolderItemsByKeychain, MetaTypeClassifier, \
- IdAttribute, FilePData, OFSProperties
+ IdAttribute, FilePData, OFSProperties, UserFolderSerializer, \
+ PythonScriptSerializer, ZSQLMethodSerializer, \
+ ZSQLMethodPropertiesSerializer
from zodb.public import RemainingState
@@ -43,11 +45,11 @@
root_mapper.setSerializer(s)
root_mapper.setGateway(ObjectGateway()) # No storage necessary
+ # Prepare some common names
if stored_keychains:
folder_items_aspect = FolderItemsByKeychain()
else:
folder_items_aspect = FolderItems()
-
properties_aspect = OFSProperties()
# folder mapper
@@ -93,12 +95,64 @@
classifier.register('Page Template', 'ZopePageTemplate', (
'.pt', '.zpt', '.html'))
+ # dtml method mapper
+ m = root_mapper.addSubMapper('OFS.DTMLMethod.DTMLMethod')
+ s = ObjectSerializer('OFS.DTMLMethod', 'DTMLMethod')
+ m.setSerializer(s)
+ s.addAspect('text', StringDataAttribute('raw'))
+ s.addAspect('id', IdAttribute())
+ s.addAspect('remainder', RemainingState())
+ classifier.register('DTML Method', 'OFS.DTMLMethod.DTMLMethod', (
+ '.dtml', ))
+
+ # dtml document mapper
+ m = root_mapper.addSubMapper('OFS.DTMLDocument.DTMLDocument')
+ s = ObjectSerializer('OFS.DTMLDocument', 'DTMLDocument')
+ m.setSerializer(s)
+ s.addAspect('text', StringDataAttribute('raw'))
+ s.addAspect('properties', properties_aspect)
+ s.addAspect('id', IdAttribute())
+ s.addAspect('remainder', RemainingState())
+ classifier.register('DTML Document', 'OFS.DTMLDocument.DTMLDocument', (
+ '.dtml', ))
+
+ # zsql mapper
+ m = root_mapper.addSubMapper('Products.ZSQLMethods.SQL.SQL')
+ s = ObjectSerializer('Products.ZSQLMethods.SQL', 'SQL')
+ m.setSerializer(s)
+ s.addAspect('text', ZSQLMethodSerializer())
+ s.addAspect('properties', ZSQLMethodPropertiesSerializer())
+ s.addAspect('id', IdAttribute())
+ s.addAspect('remainder', RemainingState())
+ classifier.register('Z SQL Method', 'Products.ZSQLMethods.SQL.SQL', (
+ '.sql', ))
+
+ # python script mapper
+ m = root_mapper.addSubMapper('PythonScript') # abbreviated name
+ s = ObjectSerializer('Products.PythonScripts.PythonScript',
+ 'PythonScript')
+ m.setSerializer(s)
+ s.addAspect('body', PythonScriptSerializer())
+ s.addAspect('id', IdAttribute())
+ s.addAspect('remainder', RemainingState())
+ classifier.register('Script (Python)',
+ 'PythonScript', ('.py', ))
+
+ # user folder mapper
+ m = root_mapper.addSubMapper('AccessControl.User.UserFolder')
+ s = ObjectSerializer('AccessControl.User', 'UserFolder')
+ m.setSerializer(s)
+ s.addAspect('data', UserFolderSerializer())
+ s.addAspect('id', IdAttribute())
+ s.addAspect('remainder', RemainingState())
+ classifier.register('User Folder', 'AccessControl.User.UserFolder')
+
# anyfolder mapper
m = root_mapper.addSubMapper('anyfolder')
s = AnyObjectSerializer()
m.setSerializer(s)
s.addAspect('items', folder_items_aspect)
- #s.addAspect('properties', properties_aspect)
+ s.addAspect('properties', OptionalAspect(properties_aspect, []))
s.addAspect('id', IdAttribute())
s.addAspect('remainder', RemainingState())
classifier.registerDefaultStorage('(folderish object)', 'anyfolder', 1)
=== Products/AdaptableStorage/Zope2SQL.py 1.7.2.1 => 1.7.2.2 ===
--- Products/AdaptableStorage/Zope2SQL.py:1.7.2.1 Sat Jan 4 14:09:18 2003
+++ Products/AdaptableStorage/Zope2SQL.py Tue Jan 21 03:11:11 2003
@@ -18,17 +18,16 @@
from mapper.public import ObjectGateway
-from gateway_sql.public import PsycopgConnection, SQLClassification, \
+from gateway_sql.public import SQLClassification, \
SQLFolderItems, SQLItemId, SQLKeychainGenerator, SQLObjectData, \
- SQLProperties, SQLRemainder
+ SQLProperties, SQLRemainder, SQLUserList
from Zope2Mapper import createZope2Mapper
-def createMapperPlus(params='', table_prefix='zodb', volatile=1):
+def createSQLMapper(conn, volatile=1):
"""Object mapper factory, with extra return arg for testing purposes
"""
- conn = PsycopgConnection(params, table_prefix)
root_mapper = createZope2Mapper(0, 1)
root_mapper.setVolatile(int(volatile))
root_mapper.getClassifier().setGateway(SQLClassification(conn))
@@ -78,10 +77,17 @@
# image gateway is identical
root_mapper.getSubMapper('OFS.Image.Image').setGateway(g)
+ # user folder gateway
+ g = ObjectGateway()
+ g.addGateway('data', SQLUserList(conn))
+ g.addGateway('id', item_id_gw)
+ g.addGateway('remainder', remainder_gw)
+ root_mapper.getSubMapper('AccessControl.User.UserFolder').setGateway(g)
+
# anyfolder object gateway
g = ObjectGateway()
g.addGateway('items', folder_items_gw)
- #g.addGateway('properties', properties_gw)
+ g.addGateway('properties', properties_gw)
g.addGateway('id', item_id_gw)
g.addGateway('remainder', remainder_gw)
root_mapper.getSubMapper('anyfolder').setGateway(g)
@@ -99,14 +105,18 @@
g.addGateway('remainder', remainder_gw)
root_mapper.getSubMapper('OFS.Application.Application').setGateway(g)
+ root_mapper.checkConfiguration()
+
return root_mapper, [conn], gws
-def createMapper(params='', table_prefix='zodb', volatile=1):
+def createPostgreSQLMapper(params='', table_prefix='zodb', volatile=1):
"""Object mapper factory.
Usage in database configuration file:
factory=Products.AdaptableStorage.Zope2SQL.createMapper
"""
- return createMapperPlus(params, table_prefix, volatile)[:2]
+ from gateway_sql.public import PsycopgConnection
+ conn = PsycopgConnection(params, table_prefix)
+ return createSQLMapper(conn, volatile)[:2]
=== Products/AdaptableStorage/doc.txt 1.4 => 1.4.2.1 ===
--- Products/AdaptableStorage/doc.txt:1.4 Fri Jan 3 17:04:14 2003
+++ Products/AdaptableStorage/doc.txt Tue Jan 21 03:11:11 2003
@@ -1,9 +1,9 @@
AdaptableStorage is a framework for mapping Python objects to
arbitrary databases and formats. It is based on a system of
-interchangeable components that can be extended or replaced. It is an
-enhancement for ZODB, but its components can also be used
-independently of ZODB.
+interchangeable components that can be extended or replaced. It
+enhances ZODB, but its components can also be used independently of
+ZODB.
At this time, AdaptableStorage comes with two default mappings, a
Zope2-filesystem mapping and a Zope2-PostgreSQL mapping. These
@@ -52,15 +52,15 @@
or change the way objects are stored.
-Setting Up a SQL Mapping
+Setting Up a Relational Database Mapping
-Install PostgreSQL and create a database. Add the following
-dbtab.conf entries:
+Install PostgreSQL and psycopg and create a database. Add the
+following dbtab.conf entries:
[Storage: SQL]
type=ASStorage
-factory=Products.AdaptableStorage.Zope2SQL.createMapper
+factory=Products.AdaptableStorage.Zope2SQL.createPostgreSQLMapper
params=
volatile=1
@@ -71,8 +71,8 @@
auto_create=1
-Add to the "params" line the parameters for connecting to the
-database, such as the host name, user name, database name, and
+Add to the "params" line the necessary parameters for connecting to
+the database, such as the host name, user name, database name, and
password. (If Postgres is installed locally and you create a default
database, you may be able to leave params blank.) When you add the
"/sql" mount point, Zope will create some tables with the prefix
=== Products/AdaptableStorage/patches.py 1.2 => 1.2.2.1 ===
--- Products/AdaptableStorage/patches.py:1.2 Wed Dec 25 00:27:35 2002
+++ Products/AdaptableStorage/patches.py Tue Jan 21 03:11:11 2003
@@ -18,10 +18,13 @@
from __future__ import nested_scopes
-from DateTime import DateTime
from cPickle import Pickler, Unpickler
from cStringIO import StringIO
+
from Acquisition import aq_base
+from DateTime import DateTime
+
+from zodb.utils import copyOf
def applyPersistentExtraPatch():
@@ -52,30 +55,28 @@
try:
return self._real_getCopy(container)
except NotImplementedError:
- last_ghost = None
- def persistent_id(ob):
- if getattr(ob, '_p_changed', 0) is None:
- # Load temporarily
- if last_ghost is not None:
- last_ghost._p_changed = None
- last_ghost = ob
- ob._p_changed = 0
- return None
- stream = StringIO()
- p = Pickler(stream, 1)
- p.dump(aq_base(self))
- if last_ghost is not None:
- last_ghost._p_changed = None
- stream.seek(0)
- u = Unpickler(stream)
- ob = u.load()
- return ob
+ return copyOf(aq_base(self))
from OFS.CopySupport import CopySource
CopySource._real_getCopy = CopySource._getCopy
CopySource._getCopy = _getCopy
+def applySetObPatch():
+ # Fall back to copying when move/rename is not possible.
+ def _setOb(self, id, object):
+ if getattr(self, '_use_fixed_oids_', 0):
+ if object._p_oid is not None:
+ old = object
+ # Forget changes to the original object
+ old._p_changed = 0
+ object = copyOf(object)
+ setattr(self, id, object)
+
+ from OFS.ObjectManager import ObjectManager
+ ObjectManager._setOb = _setOb
+
+
def applyTmpStorePatch():
from TmpStore import TmpStore as patched_TmpStore
import ZODB.TmpStore
@@ -84,5 +85,6 @@
def applyPatches():
applyPersistentExtraPatch()
applyCopySupportPatch()
+ applySetObPatch()
applyTmpStorePatch()
=== Products/AdaptableStorage/version.txt 1.4 => 1.4.2.1 ===
--- Products/AdaptableStorage/version.txt:1.4 Tue Dec 31 16:56:08 2002
+++ Products/AdaptableStorage/version.txt Tue Jan 21 03:11:11 2003
@@ -1 +1 @@
-AdaptableStorage-0.3+
+AdaptableStorage-0.4.1