I'm developing an intranet document management system with Zope/Plone and I'm concerned about the capabilities of ZODB. The system is going to store many thousands of large files, so the Data.fs is very likely to grow up to tens of GB. I'm wondering if ZODB can handle such amount of data in its standard implementation (using Data.fs). I investigated several alternatives, including ApeLib/Dbtab, but they don't seem to work out of the box with Plone (while thay do in ZMI). Is somebody aware of **fully-tested, real-world** applications with multi-GB storage or, in other words, should I stay with ZODB or search for alternatives? Thanks Alessio Beltrame
I'm developing an intranet document management system with Zope/Plone and I'm concerned about the capabilities of ZODB. The system is going to store many thousands of large files, so the Data.fs is very likely to grow up to tens of GB. I'm wondering if ZODB can handle such amount of data in its standard implementation (using Data.fs). I investigated several alternatives, including ApeLib/Dbtab, but they don't seem to work out of the box with Plone (while thay do in ZMI).
You can use DirectoryStorage http://dirstorage.sf.net that is a drop-in replacement for FileStorage. It uses one file per object version
On Jan 23, 2004, at 12:14 PM, azazel wrote:
You can use DirectoryStorage http://dirstorage.sf.net that is a drop-in replacement for FileStorage. It uses one file per object version
This looks promising. I've downloaded and installed it, but I don't understand the script for copying data from FileStorage into DirectoryStorage. From where is that script supposed to be run? I tried running it from my Zope directory, using the Zope python, and got errors. Running interactively, it chokes on the first line:
from ZODB.FileStorage import FileStorage Traceback (most recent call last): File "<stdin>", line 1, in ? ImportError: No module named ZODB.FileStorage
What am I (obviously) missing? ___/ / __/ / ____/ Ed Leafe Linux Love: unzip;strip;touch;finger;mount;fsck;more;yes;umount;sleep
On Mon, Jan 26, 2004 at 11:06:18AM -0500, Ed Leafe wrote:
On Jan 23, 2004, at 12:14 PM, azazel wrote:
You can use DirectoryStorage http://dirstorage.sf.net that is a drop-in replacement for FileStorage. It uses one file per object version
This looks promising. I've downloaded and installed it, but I don't understand the script for copying data from FileStorage into DirectoryStorage. From where is that script supposed to be run? I tried running it from my Zope directory, using the Zope python, and got errors. Running interactively, it chokes on the first line:
from ZODB.FileStorage import FileStorage Traceback (most recent call last): File "<stdin>", line 1, in ? ImportError: No module named ZODB.FileStorage
What am I (obviously) missing?
you probably need to do: export PYTHONPATH=zope-src/lib/python ... where zope-src is the zope source directory. there are lots of zodb scripts that require this, and it seems to be an unspoken rule that users should figure it out the hard way ;-) -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's CARDIO-FLOWER WITICIST! (random hero from isometric.spaceninja.com)
On Jan 26, 2004, at 11:27 AM, Paul Winkler wrote:
you probably need to do: export PYTHONPATH=zope-src/lib/python ... where zope-src is the zope source directory.
there are lots of zodb scripts that require this, and it seems to be an unspoken rule that users should figure it out the hard way ;-)
OK, thanks, that did the trick. I now have tons of files in my DS directory. Now how do I tell Zope to use DirectoryStorage instead of FileStorage? I restarted Zope and modified a few things, but according to the timestamps, my Data.fs was the only thing that changed. I've looked through the docs, and cannot see what I'm doing wrong. ___/ / __/ / ____/ Ed Leafe Linux Love: unzip;strip;touch;finger;mount;fsck;more;yes;umount;sleep
On Monday 26 January 2004 21:54, Ed Leafe wrote:
Now how do I tell Zope to use DirectoryStorage instead of FileStorage?
This is step 5 in doc/install in your DirectoryStorage distribution, also at http://dirstorage.sourceforge.net/install.html -- Toby Dickenson
On Jan 27, 2004, at 3:06 AM, Toby Dickenson wrote:
Now how do I tell Zope to use DirectoryStorage instead of FileStorage?
This is step 5 in doc/install in your DirectoryStorage distribution, also at http://dirstorage.sourceforge.net/install.html
I think I see the problem now. I use the -u switch to start Zope, and DS isn't compatible with that. Guess I have to overhaul my Zope installation before I think about switching. ___/ / __/ / ____/ Ed Leafe http://leafe.com/ http://opentech.leafe.com
On Tuesday 27 January 2004 15:09, Ed Leafe wrote:
On Jan 27, 2004, at 3:06 AM, Toby Dickenson wrote:
Now how do I tell Zope to use DirectoryStorage instead of FileStorage?
This is step 5 in doc/install in your DirectoryStorage distribution, also at http://dirstorage.sourceforge.net/install.html
I think I see the problem now. I use the -u switch to start Zope, and DS isn't compatible with that. Guess I have to overhaul my Zope installation before I think about switching.
That *might* not be true any more. Chris M did some work on Zope 2.7 to make sure that Zope doesnt open the storage until after it has dropped root. Ive not tested that with DirectoryStorage, but it looks good in theory. -- Toby Dickenson
On Tue, Jan 27, 2004 at 10:09:07AM -0500, Ed Leafe wrote:
On Jan 27, 2004, at 3:06 AM, Toby Dickenson wrote:
Now how do I tell Zope to use DirectoryStorage instead of FileStorage?
This is step 5 in doc/install in your DirectoryStorage distribution, also at http://dirstorage.sourceforge.net/install.html
I think I see the problem now. I use the -u switch to start Zope, and DS isn't compatible with that. Guess I have to overhaul my Zope installation before I think about switching.
-u is evil anyway. Last time i checked, it didn't work at all with ZEO. IMO there are better ways to present zope on privileged ports: * use a reverse proxy (pound, apache, squid). There are plenty of other good reasons to do this, not least HTTP sanitization. * configure your system to forward packets on the privileged port to the non-privileged zope port. Our sysadmin did this on Linux using ipchains when we needed to present zope's ftp port on port 21. The only limitation we encountered is that you cannot test it on localhost. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's STUPENDOUS PERFORMANCE ARTIST! (random hero from isometric.spaceninja.com)
On Jan 27, 2004, at 11:15 AM, Paul Winkler wrote:
-u is evil anyway. Last time i checked, it didn't work at all with ZEO. IMO there are better ways to present zope on privileged ports:
IIRC, that's the way Zope came when I installed it about 1.5 years ago (2.6.0). It's been working, so I haven't messed around with it at all. I suppose it's about time to think about upgrading to 2.7 - it's just a matter of finding the available time to do it. ___/ / __/ / ____/ Ed Leafe Linux Love: unzip;strip;touch;finger;mount;fsck;more;yes;umount;sleep
participants (5)
-
azazel -
Ed Leafe -
Paul Winkler -
Toby Dickenson -
WebMaster＠solari.it