[Zope] large images to a database via zope.

[ghaley@mail.venaca.com]

hi,

thank you, phillippe, for your thoughtful comments.  

the concern about the size of the data.fs is exactly why we have used an
external db to store most of the content for our site.  i have worried
about the integrity of the binary data being moved in and out of a db,
especially the larger data.  things below a meg or so don't seem to be
affected at all, but i've not done enough testing of the files that are
split then retrieved and rejoined to be able to state with the same
confidence that they are or are not getting corrupted.

one of the other concerns about storing on a file system was that we would
get so many files that the inode tables would get blown up.  a single db
table is 3 files (under mysql, anyway), and the thinking was that this
would shield the inode table from too much use.  

perhaps there is some other experience with this sort of thing, a site
with, let's say, a million files.  has anyone noticed problems storing
that much data?  i'm certain it's been done, but i don't know about the
strategies involved.

ciao!
greg.

Gregory Haley
DBA/Web Programmer
Venaca, LLC.

> More generally, the advice would be not to store large files of any kind in
> a db.
> 
> This makes me thing about this :
> 
> It must be the same in zope : it must take more time to extract some data
> from a large file in the filesystem (the data.fs) instead of storing a link
> to it in the db, and store it on the filesystem. I guess the filesystem is
> allways the fastest way to retrieve data (for example because it uses
> caching, raid optimizations...)
> 
> What would be a really cool feature in zope is a new behavior of the zodb :
> have the admin set a "maximum file size" to be stored in the db. If a file
> uploaded to zope is larger, it would be stored automatically on the
> filesystem. This decision could also be based on datatypes.
> 
> I personnaly do not trust this much a big binary file. For instance it's a
> lot harder to make a backup of a large 100 Mb file than several smaller
> files. You would only backup modified parts. Often the larger files are
> never modified because they are big images / downloads.
> 
> 
> Philippe Jadin (who hopes it's not a stupid idea)
> 
> 
> 
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - 
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope-dev )
> 
>