[Zope] DBTab product questions
Thierry FLORAC
thierry.florac@onf.fr
Tue, 18 Mar 2003 11:51:55 +0100
--Boundary-00=_Lpvd+ZtuEwNotnA
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Hi,
I've just tried to install and setup the DBTab product on my existing
Zope-2.6.1 server ; my goal is to be able to split my Zope database into
several parts, to be able to manage them more easilly (for backups,...).
So I'd like to have three FileStorage databases, called :
- Users, to store my users properties (mount point: /intraforet/users)
- Catalog, to store my catalog (mount point: /intraforet/if_catalog)
- Main, to store all the other objects (mount point: /).
"/intraforet" is a folder used as a VirtualSiteRoot for my site ; you'll find
my corresponding dbtab.conf file joined to this mail.
My problem is that the corresponding storages are created at startup, but
there's never any object which is stored into it ; I've tried to delete and
recreate my mount points, without any result. So here are my questions :
- did I miss anything ?
- can I setup DBTab for an already existing Zope site ?
- can I use DBTab to store specific classes, like a whole ZCatalog ?
- does the use of DBTab require any specific attention while writing Python
products ?
Thanks for any help.
Thierry
--
Linux every day, keeps Dr Watson away...
http://gpc.sourceforge.net -- http://www.ulthar.net
--Boundary-00=_Lpvd+ZtuEwNotnA
Content-Type: application/octet-stream;
name="dbtab.conf"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="dbtab.conf"
######################################################################
##
## This file specifies what databases Zope should use. It gets read
## once at startup by the DBTab product. To activate a new
## database configuration, you need to restart Zope.
##
## dbtab.conf.in is the default configuration. Instead of modifying
## it directly, copy it to your INSTANCE_HOME or root directory of
## your Zope installation and call it "dbtab.conf". Then the DBTab
## product will read dbtab.conf instead of dbtab.conf.in. Also,
## symlink or copy Products/DBTab/custom_zodb.py to your INSTANCE_HOME
## or the root of your Zope software, so that Zope will load the
## database mounted at "/" as its main database.
##
## The configuration below shows the most basic setup. It corresponds
## closely with a default Zope install. It specifies that Zope data
## should be stored in one large file called Data.fs and up to 4000
## objects should be kept in RAM per connection.
##
######################################################################
[Storage: Main]
type=FileStorage
file_name=%(CLIENT_HOME)s/Data.fs
[Database: Main]
cache_size=10000
mount_paths=/
[Storage: Users]
type=FileStorage
file_name=%(CLIENT_HOME)s/Users.fs
[Database: Users]
cache_size=1000
mount_paths=/intraforet/users
open_at_startup=1
[Storage: Catalog]
type=FileStorage
file_name=%(CLIENT_HOME)s/Catalog.fs
[Database: Catalog]
cache_size=10000
mount_paths=/intraforet/if_catalog
open_at_startup=1
######################################################################
##
## Here is an example of a simple ZEO client to replace the above
## configuration. In this example, the client connects to a ZEO
## storage server on port 9001 of the same box. Addresses to connect
## to the ZEO storage server can either be ZEO URLs, as shown below,
## or paths to Unix domain sockets.
##
######################################################################
# [Storage: Main]
# type=ClientStorage
# addr=zeo://localhost:9001
# [Database: Main]
# cache_size=4000
# mount_paths=/
######################################################################
##
## You can load as many databases as you need, as long as each
## database has a different name and is mounted at non-conflicting
## paths. Below is an example of a simple mounted wiki database.
##
## The 'open_at_startup' option causes DBTab to open the database
## during Zope initialization. If it fails to connect, Zope will stop
## immediately. This is useful for important mounts.
##
## After adding a mount point to the configuration file, restart Zope
## and visit the Zope management interface. Add a "DBTab Mount
## Point", which displays a form for all configured mount points. If
## there are no conflicts, click the "add selected mount points"
## button to finalize the creation of the mount points.
##
######################################################################
# [Storage: Wikis]
# type=FileStorage
# file_name=%(CLIENT_HOME)s/Wikis.fs
# [Database: Wikis]
# cache_size=4000
# mount_paths=/wikis
# open_at_startup=1
######################################################################
##
## Here is an example of a ZEO client mounting a ZEO database.
## Mounting is a lot like NFS. Each ZEO client must have its own
## connection to the server. A ZEO server can not mount on behalf of
## its clients. Also, fine-grained security is implemented by the ZEO
## client, not the ZEO storage server, so all ZEO clients must be
## highly trusted.
##
######################################################################
# [Storage: Main-Client]
# type=ClientStorage
# addr=zeo://localhost:9001
# [Database: Main-Client]
# cache_size=4000
# mount_paths=/domain_folder; /group1
# open_at_startup=1
######################################################################
##
## Here is an example of mounting the temporary folder for session
## data using DBTab instead of the default mechanisms. You can
## experiment with this to share sessions using ZEO. Note that you'll
## have to delete the temp_folder object before you can add the mount
## point in the management interface, and you can't do this the normal
## way since the management interface prevents you from removing the
## temp_folder. The following external method gets around the
## restriction:
##
## def remove_temp_folder(self):
## self.getPhysicalRoot()._delObject('temp_folder')
## return 'Removed temp_folder successfully.'
##
## TemporaryStorage is a storage type designed for storing session
## data. It is located in the TemporaryFolder product that comes with
## Zope. According to its description, TemporaryStorage "uses RAM to
## persist objects, much like MappingStorage. Unlike MappingStorage,
## it needs not be packed to get rid of non-cyclic garbage and it does
## rudimentary conflict resolution."
##
## The LowConflictConnection class helps prevent conflicts in session
## data. According to its description, "This is potentially dangerous
## for applications that need absolute consistency, but sessioning is
## not one of those." So be careful.
##
######################################################################
# [Storage: Sessions]
# type=TemporaryStorage
# [Database: Sessions]
# connection_class=LowConflictConnection
# mount_paths=/temp_folder
# auto_create=1
######################################################################
##
## The example below documents all available parameters for a
## FileStorage configuration.
##
######################################################################
# [Storage: Example]
## The type parameter specifies the class of the storage. It is
## required. Alternatives are mentioned later.
# type=FileStorage
## file_name specifies the full path to the database file. It is
## required for FileStorages. ZODB also creates files next to the
## database file with the suffixes ".tmp", ".index", ".index_tmp",
## ".lock", ".old", and ".pack". Note that the file_name parameter is
## often a good place to use environment variable substitutions using
## Python string interpolation syntax like "%(CLIENT_HOME)s".
# file_name=%(CLIENT_HOME)s/Data.fs
## If the create flag is set to 1, and a database file already exists,
## ZODB deletes the database file and creates a new one. This
## parameter defaults to 0. Most sites should not set this.
# create=0
## If the read_only flag is set to 1, ZODB opens the database file in
## read-only mode, preventing Zope from writing to the database. This
## parameter defaults to 0.
# read_only=0
## The quota parameter prevents the size (in bytes) of a FileStorage
## from expanding beyond a specified decimal limit. 0 disables quota
## checking.
# quota=0
## The stop parameter lets you read the database up to the transaction
## that begins at a specified decimal offset, allowing you to read a
## former database state. You must open the database in read-only
## mode. This is an advanced option. 0 disables stop mode.
# stop=0
######################################################################
##
## The example below documents all available parameters for a
## database configuration.
##
######################################################################
# [Database: Example]
## Note that the database name (in square brackets) has to match a
## storage name.
## Setting the open_at_startup flag to 1 causes DBTab to open the
## database during Zope initialization. If it fails to connect, Zope
## will stop immediately. This is useful for important mounts. This
## option defaults to 0.
# open_at_startup=0
## The mount_paths parameter configures mount points. The DBTab
## convention for mount points is slightly different from Unix
## conventions: in Unix, you normally mount a whole filesystem, but
## when using DBTab in Zope, you normally mount only a part of a
## database. Here is a simple example:
# mount_paths=/News/Archives
## This parameter loads an object located at /News/Archives from the
## mounted database and makes it available as an object called
## Archives in the News folder of the main database. The root of
## the mounted database is thus inaccessible in this ZEO client.
## You can set up multiple mount points into a single database by
## separating the paths with semicolons.
# mount_paths=/News/2001; /News/2000; /News/Old
## Use a colon after the mount path to specify the real path in the
## mounted database. (Otherwise, the real path is the same as the
## mount path.) Using this syntax, you can coerce DBTab into using
## Unix mounting conventions, but it is not recommended. To mount the
## root of the mounted database into a folder in the main database:
# mount_paths=/News/Archives:/
## This loads the OFS.Application object at the root of the mounted
## database and makes it available as /News/Archives. But when an
## Application object exists anywhere besides the root of your Zope
## site, your site is likely to break. If you're familiar with ZODB,
## you can programmatically arrange for the root object of the mounted
## database to be some other kind of object. But normally you just
## don't need to.
## On the other hand, if you're writing an external ZODB application
## and you want to mount your application into Zope, you may not have
## an Application at the root of your mounted ZODB at all. The true
## root of a ZODB is a PersistentMapping where Zope puts two keys,
## "Application" and "ZGlobals". If the important part of your ZODB
## is stored under a different key, such as "Calendar", you can use a
## tilde at the beginning of the mounted path to specify an
## alternative root name. Example:
# mount_paths=/MyApp/MyCalendar:~Calendar/
## The auto_create parameter is useful for volatile data storage such
## as DemoStorage or MappingStorage. Since the database gets wiped
## out when the process stops, the mountable object needs to be
## created again upon restart. auto_create does this automatically.
# auto_create=0
## The class_factory parameter selects what kind of class factory to
## use in the database. It exists to allow ZODB to handle missing
## class instances gracefully and to support ZClasses. The default is
## "auto", which is good enough for almost everyone.
## - minimal: No handling of missing classes and no ZClass support.
## If ZODB attempts to load an instance of class it can't find, even
## as a side effect, ZODB will raise an exception that propagates to
## the application. Standalone ZODB applications usually have this
## behavior.
## - simple: Instances of missing classes are replaced with
## "OFS.Broken" instances; no ZClass support.
## - zope: OFS.Broken and ZClass support.
## - auto: Same as the "zope" class factory with the added feature
## that ZClasses instances in mounted databases are loaded from the
## parent or root database rather than the mounted database.
# class_factory=auto
## The 'class' parameter lets you use a class other than ZODB.DB.DB
## for your database (usually a subclass of DB). This is an advanced
## option.
# class=ZODB.DB.DB
## The 'connection_class' parameter lets you use a class other than
## ZODB.Connection.Connection for the database connections (usually a
## subclass of Connection). This is useful for session data storage,
## for example, where the LowConflictConnection class helps reduce
## database conflicts.
# connection_class=ZODB.Connection.Connection
## The pool_size parameter sets the number of allowable concurrent database
## connections. Since each connection has its own cache of recently used
## objects in the database, this number should be kept low, but it should
## be at least as high as the number of threads. The default is 7.
# pool_size=7
## The cache_size parameter sets the maximum number of objects to keep
## in the cache for each connection. This should be set high enough
## that Zope doesn't have to load objects on every request, but not so
## high that Zope consumes excessive RAM. The default is 5000. Watch
## the Database Activity page in the control panel to see whether this
## number is right for your site.
# cache_size=5000
## The cache_deactivate_after parameter is no longer in use (as of
## Zope 2.6 and ZODB 3.1). It once set the minimum time (in seconds)
## that cached objects should be kept in RAM. The default is 60.
# cache_deactivate_after=60
## The version_pool_size, version_cache_size, and
## version_cache_deactivate_after parameters control caching of
## objects in Zope Versions. Tune them if you use Zope Versions.
## (version_cache_deactivate_after is no longer used as of Zope 2.6.)
# version_pool_size=3
# version_cache_size=100
# version_cache_deactivate_after=10
######################################################################
##
## The example below documents all available parameters for a
## ZEO client configuration.
##
######################################################################
# [Storage: Example]
# type=ClientStorage
## The required addr parameter specifies addresses of ZEO storage
## servers. Use ZEO URL syntax to specify a host and port, or use a
## filename to specify a unix domain socket. Note that a port is
## required. An example of a ZEO URL:
# addr=zeo://localhost:5432
## You can use a Unix domain socket to connect a ZEO client and a ZEO
## storage server, as long as the OS supports it and both the client
## and the server are running on the same box (or you have some kind
## of tunnel). See the ZEO documentation for instructions on making
## the ZEO server listen on a Unix socket. Once the server is set up,
## here is how you would set up the client:
# addr=/var/zope/zeo.socket
## To use multiple addresses in a ZRS (Zope Replicated Storage)
## configuration, separate the storage server addresses with
## semicolons. The first address will be tried first.
# addr=zeo://zrs1.example.com:5432; zeo://zrs2.example.com:5432
## The optional 'storage' parameter specifies the name of the storage
## to open from the ZEO server. A ZEO server can theoretically serve
## multiple storages, though the feature is not used often. By
## default, ZEO clients load the storage called "1". This property is
## distinct from the name you assign a storage in the square brackets.
# storage=1
## The optional 'cache_size' parameter specifies the limit on the
## total size (in bytes) of all ZEO cache files. The ZEO cache is
## stored on disk and supplements the ZODB cache. This parameter
## defaults to 20000000 (about 20 MB).
# cache_size=20000000
## The 'client' parameter specifies whether to keep a persistent ZEO
## cache, and if so, what name to use to disambiguate this cache from
## other caches. If the cache is persistent (client parameter is
## nonempty), ZEO creates some ".zec" files in the "var" directory or
## CLIENT_HOME and re-validates the cache upon restart. If the cache
## is not persistent, ZEO instead uses temporary files that get
## deleted when Zope stops. By default, the cache is not persistent.
# client=
## The 'var' parameter specifies where to store the persistent ZEO
## cache. It defaults to the value of the CLIENT_HOME environment
## variable, which is usually the "var" directory of your ZEO client.
# var=%(CLIENT_HOME)s
## The 'min_disconnect_poll' parameter configures the minimum time to
## wait between failed ZEO server connection attempts. It defaults to
## 5 seconds.
# min_disconnect_poll=5
## The 'max_disconnect_poll' parameter configures the maximum time to
## wait between failed ZEO server connection attempts. When the first
## connection attempt fails, the ZEO client waits the number of
## seconds specified by min_disconnect_poll. After each subsequent
## attempt, the client waits twice as long as the previous attempt, up
## to the value of max_disconnect_poll. This parameter defaults to
## 300 seconds (5 minutes).
# max_disconnect_poll=300
## If the 'wait' flag is set to 1, ZEO will finish initialization and
## return control to Zope until it has successfully connected to a ZEO
## server. This parameter defaults to 0.
# wait=0
## If the 'read_only' flag is set to 1, ZEO will only attempt
## read-only connections to the ZEO servers. This parameter defaults
## to 0.
# read_only=0
## 'read_only_fallback' is a flag indicating whether a read-only
## remote storage should be acceptable as a fallback when no writable
## storages are available. Defaults to false. At most one of
## read_only and read_only_fallback should be true.
# read_only_fallback=0
######################################################################
##
## If you have BDBStorage installed, here is how to set up a
## Berkeley-backed storage. BDBStorage, available in ZODB version
## 3.2 and above, is the successor to bsddb3storage. (A lot of these
## directions are copied from BerkeleyBase.py.)
##
## BDBStorage provides two variants: BDBFullStorage and
## BDBMinimalStorage. BDBFullStorage provides undo and versioning,
## while BDBMinimalStorage does not. Choose which one to use in the
## "type" field.
##
######################################################################
# [Storage: BDB]
# type=BDBFullStorage # or BDBMinimalStorage
## Optional env, if given, is a string that names the database
## environment, i.e. essentially the name of a directory into which
## BerkeleyDB will store all its supporting files. It is passed
## directly to DbEnv().open(), which in turn is passed to the
## BerkeleyDB function DBEnv->open() as the db_home parameter.
##
## Note that if you want to customize the underlying Berkeley DB
## parameters, this directory can contain a DB_CONFIG file as per the
## Sleepycat documentation.
# env=
## Optional prefix is the string to prepend to name when passed to
## DB.open() as the dbname parameter. In other words, (prefix +
## storage name) is passed to the BerkeleyDb function DB->open() as
## the database parameter. It defaults to "zodb_".
# prefix=zodb_
## Berkeley storages need to be checkpointed occasionally, otherwise
## automatic recovery can take a huge amount of time. You should set
## up a checkpointing policy which trades off the amount of work done
## periodically against the recovery time. Note that the Berkeley
## environment is automatically, and forcefully, checkpointed twice
## when it is closed.
##
## interval indicates how often, in seconds, a Berkeley checkpoint is
## performed. If this is non-zero, checkpointing is performed by a
## background thread. Otherwise checkpointing will only be done when
## the storage is closed. You really want to enable checkpointing. ;
# interval=120
## kbytes is passed directly to txn_checkpoint().
# kbyte=0
## min is passed directly to txn_checkpoint().
# min=0
## You can acheive one of the biggest performance wins by moving the
## Berkeley log files to a different disk than the data files. We saw
## between 2.5 and 7 times better performance this way. If not None,
## logdir is passed to the environment's set_lg_dir() method before it
## is opened.
# logdir=
## You can also improve performance by tweaking the Berkeley cache
## size. Berkeley's default cache size is 256KB which is usually too
## small. The default cache size is 128MB which seems like a useful
## tradeoff between resource consumption and improved performance.
## You might be able to get slightly better results by turning up the
## cache size, although be mindful of your system's limits. See here
## for more details:
##
## http://www.sleepycat.com/docs/ref/am_conf/cachesize.html
##
## cachesize should be the size of the cache in bytes.
# cachesize=135000000
## These attributes control the autopacking thread: frequency is
## the time in seconds after which an autopack phase will be
## performed. E.g. if frequency is 3600, an autopack will be done
## once per hour. Set frequency to 0 to disable autopacking (the
## default).
# frequency=0
## packtime is the time in seconds marking the moment in the past at
## which to autopack to. E.g. if packtime is 14400, autopack will
## pack to 4 hours in the past. For Minimal storage, this value is
## ignored.
# packtime=14400
## classicpack is an integer indicating how often an autopack phase
## should do a full classic pack. E.g. if classicpack is 24 and
## frequency is 3600, a classic pack will be performed once per day.
## Set to zero to never automatically do classic packs. For Minimal
## storage, this value is ignored -- all packs are classic packs.
# classicpack=0
## read_only causes ReadOnlyError's to be raised whenever any
## operation (except pack!) might modify the underlying database.
# read_only=0
######################################################################
##
## You can also use custom storages not known to DBTab. Prefix the
## class name of the storage with the name of the module. The
## constructor of your class may need to convert its keyword arguments
## from strings to the correct types.
##
## For experimentation, MappingStorage and DemoStorage are available.
## DemoStorage provides undo support, while MappingStorage does not.
## Both store all data in volatile RAM, meaning all data is lost
## between Zope restarts. Neither accepts any extra parameters
## (currently). An example follows. It uses the auto_create
## parameter to set up the database on restart.
##
######################################################################
# [Storage: Volatile Demo]
# type=DemoStorage
# [Database: Volatile Demo]
# mount_paths=/
# auto_create=1
--Boundary-00=_Lpvd+ZtuEwNotnA--