[Zope] Re: ZEO and Multiple Zope Sites

[munwar_shariff]

too large Data.fs ?

There might be a problem with having the Zope Database (ZODB - 
Data.fs file) as one file and that is hitting the 2-gig file limit 
that occurs in many OS's and python.

There is an article on "Recovering from hitting the 2 gig data base 
limit" at http://www.zope.org/Members/andym/2gig

Check ZODB Roadmap at http://www.zope.org//Wikis/ZODB/ZODBRoadmap.
ZODB storages,
- FileStorage
- bsddb3Storage
- ReplicatedStorage
- OracleStorage

The first action plan should be to reduce the size of existing ZODB. 
The following are some of ways to reduce the ZODB size,


1> Pack ZODB

Go to the root folder of Zope management screens. Click 
on "Control_Panel" and click on "Database Management".  You can pack 
the database by removing previous revisions of objects.  This will 
keep a backup of database as "Data.fs.old" (at Zope installation's 
var directory) and reduces the database size.

2> External File

The ExtFile/ExtImage Product stores a file externally in the file 
system and keeps only meta data in the Zope Database. The file is 
stored in an external file repository and is accessible via its 
corresponding Zope object.  Find more information at 
http://www.zope.org/Members/MacGregor/ExtFile

3> LocalFS

The Local File System product allows you to access files from the 
server's local file system as if they were objects in the ZObject 
database. It also allows you to upload files to the file system. 

4> Storing in External Database

Consider storing your data in external databases like MySQL etc.  
There are Zope database connectors available for most of the 
databases (RDBMS).

5> Membership Data

If your site has large member base and if you are storing membership 
data in ZODB, then you can consider the following options to store 
member data,
- LDAP
- MySQL User Folder
- or other external storages

6> Break the Single Site into multiple Sites 

If your site is too large, you can as well consider to break the 
single site into multiple sites. You can have two or many Zope sites 
pointing to the same user folder (such as LDAP) for authentication.  
You can use SiteRoot to have logical URL names.  If you use 
consistent style sheets, framework and GUI, users feel that it is a 
single Zope site.

7> Select OS

Some OS's don't have this problem Solaris, Windows NT/2000, AIX, and 
FreeBSD (too name a few).  Move the db to an os that supports large 
file system. Try 64-bit Linux, where Python 2.0 supports large 
files.  Check the article "Recovering from hitting the 2 gig data 
base limit" at http://www.zope.org/Members/andym/2gig

There are other ways to have multiple ZODBs :

1> Mountable Databases

It would be desirable to mix multiple storages or multiple databases 
in the same of Zope object space. You can find an article on this at, 
http://www.zope.org//Members/jim/ZODB/MountedDatabases

2> MountedFileStorage

There is a really simple Zope Product for mounting multiple ZODB 
files onto a Zope installation. You can find more information about 
the product at 
http://www.zope.org/Members/anthony/software/MountedFileStorage/

Members of Zope Community might have solved this issue in various 
innovative ways.  If you can publish/send your solutions, I can 
consolidate all of them and publish an article on this on zope.org 
site.

Best regards,
Munwar Shariff,
CTO, Certified Talent Inc,
http://www.certifiedtalent.com


--- In zope@y..., eric.n.dunn@b... wrote:
> 
> 
> We're testing and upgrade from ZOPE 2.3.3 > ZOpe 2.5.1 with ZEO 
running
> 
> We're having good success with both with only in Test environment 
(minimal
> users)
> 
> Possible 3000 users hit Production ZServers
> 
> Possible future problem.
> 
> 1.  Web site growth will cause Data.fs to become too large
> 
> Solutions???
>  ? Mounting ZODB(s)
> 
> Bottom line is... How do I prevent a meltdown of these present web 
sites
> when/if content too much for one ZServer to handle?
> 
> Any and all suggestion will be gratefully accepted
> 
> Eric Dunn