[Zope] advice needed

[sean.upton@uniontrib.com]

Big image databases can be just that: very big (I'm used to working with
media-asset management systems for newspapers, and from that perspective, I
can tell you that such a beast is likely best not done as a BLOB setup).

I would think this suffers from scalability problems, regardless of which
table handler backend you use in MySQL.  If you have Terabytes of images,
you want to store them as files across many volumes, which means you want to
store them on disk volumes on scalable filesystems with logical volume
management and good backup procedure.  BLOBs in RDBMS won't scale with your
hardware and won't let you use simple UNIX maintenance and backup tools.
For example, try incrementally backing up half-a-dozen InnoDB table-space
files using find+cpio (hint: you can't).

I would approach this with MySQL, filesystem storage for binaries, and
DirectoryStorage for ZODB (especially if this was a massive DB, which image
DBs usually are).

Sean

-----Original Message-----
From: Martin Gebert [mailto:Murphy@members.netsolution-net.de]
Sent: Tuesday, November 05, 2002 2:27 PM
To: gabecker@indiana.edu
Cc: zope@zope.org
Subject: Re: [Zope] advice needed


Have you thought about storing the images as BLOBs in MySQL? I must 
admit I lack every experience how MySQL behaves under the load of many 
Long Objects, but in a small scale I've made good experiences till now 
with that (the concerning small application will go online the next days).
Concerning the access restrictions, IMHO this would be handled the 
easiest way by roles and (in case of externel files or ZODB storage) 
different folders, if your special problem admits such an architecture.

Martin

gabecker@indiana.edu schrieb:

>I am relatively new to Zope, but I plan to use it for an image database
>system that I need to make for scientific purposes.  What I have in mind
>is to store the actual images externally in files (since there will be
>many and large files), and the info about the image (where collected,
>owner, codes etc etc) in a database like MySQL. Zope would form the
>interface.
>
>My first question is what would be a suitable way to go from here: use
>LocalFS or ExtImage for storage of many files (think of 50,000 or so)? I
>will need to be able to differentiate between users: ie not every user
>should have access to all files.
>
>Furthermore: if there are people that have already done something similar
>I would appreciate to learn from their experience. Through a web search I
>haven't been able to find such projects, at least not based on Zope.
>
>All best,
>Gabriel
>
>  
>


_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )