On Tue, Jan 13, 2004 at 12:01:09AM +0100, Sebastien Pastor wrote:
Hi there,
I am about to finalize a Zope 2 site. It will mainly play the role of groupware app and storage server. My question is : i know i ll get quicly a big amount of PSD files (photoshop) they are usually quite heavy. Do you rekon i should store them on my server file system rather than the ZODB ? Do you see any trouble putting them all into the DB in a longer term ? (i know my python is large-file compiled).
two issues: 1) A single large FileStorage has been a bit cumbersome to maintain (backups etc). recent developments (repozo.py) make it possible to do an incremental backup. There's also DirectoryStorage which on a suitable filesystem should be able to scale to a very large database. You can also mix-and-match and have multiple databases using DBTab. 2) Performance of serving large files or images from Zope is pretty bad, much slower than a static webserver like Apache. The worst part is that a few simultaneous requests for large files will essential DOS your entire zope site until they complete. I have not identified the precise point of pain, but I know that in my experience, 5 MB is OK and 50 MB is awful. I have worked around this by using FSCacheManager (available from CVS at collective.sf.net) which does "funky caching". It is easy to set up with Apache, but is only safe to use with anonymously-viewable content. I understand that Squid can be set up to do filesystem caching while forcing authentication on each request so it would be safe with secured content, so we might do that instead at a later date.
Storing on both FS and ZODB is it an easy thing to do ?
If you want the files on the filesystem, there are various solutions depending on your needs: LocalFS, FilesystemDirectoryView, and APE. I have not evaluated these for performance with large files, except LocalFS which I know is at least as slow as normal zope File objects. I tend to assume they are all slow. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's TERRA-RANDY CHAIN MISTER! (random hero from isometric.spaceninja.com)