[Zope] Hardware for Zope + ZEO

[Paul Winkler]

On Wed, Jan 29, 2003 at 03:32:20PM +0900, Wankyu Choi wrote:
> I've been thinking:
> 
> 1. Take out as much binary data as possible via External File ( my version,
> not the ones found in Zope products page ).

i can understand doing this if you're running FileStorage and want
to avoid a single bloated Data.fs file, but since you're leaning towards
DirectoryStorage, I think there's no reason to use ExternalFile at all -
it just complicates things.

> 2. Store textual data  ( raw data + its rendered version, etc ) in MySQL.

ditto

> 3. User data goes into MySQL.

ditto
 
> once but that seemed a lot of hassle, so I decided to wait for other
> solutions like the Directory Storage.

Wait no longer. ;)
 
> If most Zope users ( including yourself ) feel Filestorage is not a solution
> for more than 20GB of data, 

hell, I don't like it for 2 GB of data.
(see recent thread "POSKeyError II: Dead by Dawn" ... I wasn't kidding
when I titled it that!)


the problem is not that FileStorage won't scale - I have no evidence
either way about that, but others have reported 100 GB (*) - 
the problem is that it's very hard to manage 
properly when it gets big.  Incremental backups are impossible, and 
repair tools (fsrecover.py) take forever to run. I can only imagine
the pain of doing this with ~100GB.

Anonther solution that hasn't been mentioned in this thread:
multiple mounted storages. Never done this, but I understand
zope.org does it (one storage for wikis, another for everything else).
http://www.zope.org/Members/jim/ZODB/MountedDatabases

Shane's DBTab allows you to mix-and-match storages, running
several at once.
http://www.my-zope.org/exp/20030116101458
http://hathaway.freezope.org/Software/DBTab


-- 

Paul Winkler
http://www.slinkp.com
Look! Up in the sky! It's THE MYSTICAL SATIRE!
(random hero from isometric.spaceninja.com)