Message: 36 Date: Fri, 23 Jan 2004 16:00:28 +0100 From: "WebMaster@solari.it" <webmaster@solari.it> Subject: [Zope] ZODB limits To: zope@zope.org Message-ID: <4011370C.1090905@solari.it> Content-Type: text/plain; charset=us-ascii; format=flowed Is somebody aware of **fully-tested, real-world** applications with multi-GB storage or, in other words, should I stay with ZODB or search for alternatives?
Well Zope corporation has experience of tens gigabytes of data in FileStorage ( Data.fs ) in ZEO enviroment, so that shouldn't be the main issue. Larger issue is how fast your system will be for uplaoding and downloading the data from ZODB. Zope sucks big rocks in handling big files, so that might bring your system down or force you to bring in more machines into cluster - or move storing and serving large files outside ZODB. -- -huima
On Fri, Jan 23, 2004 at 05:11:46PM +0200, Heimo Laukkanen wrote:
From: "WebMaster@solari.it" <webmaster@solari.it> Is somebody aware of **fully-tested, real-world** applications with multi-GB storage or, in other words, should I stay with ZODB or search for alternatives?
Well Zope corporation has experience of tens gigabytes of data in FileStorage ( Data.fs ) in ZEO enviroment, so that shouldn't be the main issue. Larger issue is how fast your system will be for uplaoding and downloading the data from ZODB. Zope sucks big rocks in handling big files, so that might bring your system down or force you to bring in more machines into cluster - or move storing and serving large files outside ZODB.
Yeah, capacity planning and cluster design is critical here. For a large data store, you really want to have multiple Zopes talking to each ZEO. You also might consider using DBTab so that you can run multiple ZEOs for each Zope--- it lets you treat ZEO servers approximately like you'd treat NFS mounted filesystems. We're running a content data store that's about 17GB right now, though it was recently packed (so it has been larger in the past). It runs on an enterprise-class Sun box, though I don't know how large that box is. (I am not a sysadmin.) srl -- Shane Landrum, Software Engineer srl@boston.com boston.com / NY Times Digital
You may want to look at Shane's APE -- http://hathaway.freezope.org/ On Fri, 23 Jan 2004, Heimo Laukkanen wrote:
Message: 36 Date: Fri, 23 Jan 2004 16:00:28 +0100 From: "WebMaster@solari.it" <webmaster@solari.it> Subject: [Zope] ZODB limits To: zope@zope.org Message-ID: <4011370C.1090905@solari.it> Content-Type: text/plain; charset=us-ascii; format=flowed Is somebody aware of **fully-tested, real-world** applications with multi-GB storage or, in other words, should I stay with ZODB or search for alternatives?
Well Zope corporation has experience of tens gigabytes of data in FileStorage ( Data.fs ) in ZEO enviroment, so that shouldn't be the main issue. Larger issue is how fast your system will be for uplaoding and downloading the data from ZODB. Zope sucks big rocks in handling big files, so that might bring your system down or force you to bring in more machines into cluster - or move storing and serving large files outside ZODB.
-- -huima
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
On Fri, Jan 23, 2004 at 05:11:46PM +0200, Heimo Laukkanen wrote:
Message: 36 Date: Fri, 23 Jan 2004 16:00:28 +0100 From: "WebMaster@solari.it" <webmaster@solari.it> Subject: [Zope] ZODB limits To: zope@zope.org Message-ID: <4011370C.1090905@solari.it> Content-Type: text/plain; charset=us-ascii; format=flowed Is somebody aware of **fully-tested, real-world** applications with multi-GB storage or, in other words, should I stay with ZODB or search for alternatives?
Well Zope corporation has experience of tens gigabytes of data in FileStorage ( Data.fs ) in ZEO enviroment, so that shouldn't be the main issue.
And if it is, I second azazel's recommendation: DirectoryStorage is pretty nice. Ours is "only" about 3 GB total, currently.
Larger issue is how fast your system will be for uplaoding and downloading the data from ZODB. Zope sucks big rocks in handling big files, so that might bring your system down or force you to bring in more machines into cluster - or move storing and serving large files outside ZODB.
Or it might force you to help fix those problems ;-) there have been some discussions on zope-dev about this issue, which I am determined to work on whenever i can find the time. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's MAILMAN SEXUAL DUCK! (random hero from isometric.spaceninja.com)
participants (4)
-
Dennis Allison -
Heimo Laukkanen -
Paul Winkler -
Shane Landrum