With 300GB you are at the point where I would think about a RDBMS as backend. I assume maintaining or packing a 300GB Data.fs might be a pain (20-30GB are already hard to handle).
You lost me there. I don't think you mean **RDBMS storage**.. Then you mean you'd think about setting up an RDMBS backend outside of ZODB? ( alert! newbie factors ;-) I've been thinking: 1. Take out as much binary data as possible via External File ( my version, not the ones found in Zope products page ). 2. Store textual data ( raw data + its rendered version, etc ) in MySQL. 3. User data goes into MySQL. Then again... why should I use Zope if I have to go to all these troubles? I already have APM set up to do just that. Given all the benefits Zope offers, it might be worth the efforts... but I'd need to heavily modify all the content types to store data outside of ZODB such as External File. I tried once but that seemed a lot of hassle, so I decided to wait for other solutions like the Directory Storage. If most Zope users ( including yourself ) feel Filestorage is not a solution for more than 20GB of data, wouldn't ZC feel the same too? It seems the stock Zope is not up to a large-scale web site. (if you call 300,000 user web site large-scale... I wouldn't but...) Some even say a coule of thousand users would be the limit. I've posted similar queries about Zope's scalibility on a number of occasions, but replies suggest "one might do this" kind of stuff. There's been no concrete answer to these queries, an answer out of real experience, not a guesstimation. That confuses me. Does that mean nobody has reached the limit using Zope? 20GB of data is so normal these days. I already have double the amount of data on my site. Guess I'll start worrying about Zope's scalibility again.... Please convince me. Anyone? One mentioned months ago that Filestorage is so robust that it could withstand most abuse I could throw at it. Okay, I believe it so. But the question still remains. Why couldn't we have a FileStorage that can split over partitions ( in multiple files, I mean, why one single golliath? ) and that has an option to turn off versioning? One might say, "It's opensource, please yourself. Write one yourself." ;-) But I just really wanted to know why ZODB guys hasn't done that. Is there a reason I'm missing? Or is it still on the to-do list? Maybe I'm assuming wrong. Would you please elaborate what you mean by 'RDBMS backend'? Do you really mean I should write my own products to use MySQL as backend bypassing ZODB , for example? Or is there an RDBMS storage ( Orcale is not an option, if you mean Oracle storage )? Thanks anyway for your comments. Best Regards, Wankyu Choi -----Original Message----- From: Andreas Jung [mailto:lists@andreas-jung.com] Sent: Wednesday, January 29, 2003 2:55 PM To: Wankyu Choi; zope@zope.org Subject: Re: [Zope] Hardware for Zope + ZEO --On Mittwoch, 29. Januar 2003 13:44 +0900 Wankyu Choi <wankyu@neoqst.com> wrote:
3. I'm looking towards the Directory Storage over File Storate for tons of reasons, the reasons you might easily guess. I'll have six 73G disks with RAID5, which means I'd have to let go 73G for storing parity information. That leaves me about 365G. At least 300G will be allocated for ZODB. Would FileStorage maintain its integrity if the db grows to 300G? What I'm worried about most is that I can't make it versionless. Directory Storage has that option. Comments?
With 300GB you are at the point where I would think about a RDBMS as backend. I assume maintaining or packing a 300GB Data.fs might be a pain (20-30GB are already hard to handle). -aj