[Zope] Hardware for Zope + ZEO

[Wankyu Choi]

> With 300GB you are at the point where I would think about a RDBMS as
backend. I assume maintaining or packing a 300GB Data.fs might be a pain
(20-30GB are already hard to handle).

You lost me there. I don't think you mean **RDBMS storage**.. Then you =
mean
you'd think about setting up an RDMBS backend outside of ZODB?

( alert! newbie factors ;-)

I've been thinking:

1. Take out as much binary data as possible via External File ( my =
version,
not the ones found in Zope products page ).
2. Store textual data  ( raw data + its rendered version, etc ) in =
MySQL.
3. User data goes into MySQL.

Then again... why should I use Zope if I have to go to all these =
troubles? I
already have APM set up to do just that. Given all the benefits Zope =
offers,
it might be worth the efforts... but I'd need to heavily modify all the
content types to store data outside of ZODB such as External File. I =
tried
once but that seemed a lot of hassle, so I decided to wait for other
solutions like the Directory Storage.

If most Zope users ( including yourself ) feel Filestorage is not a =
solution
for more than 20GB of data, wouldn't ZC feel the same too? It seems the
stock Zope is not up to a large-scale web site. (if you call 300,000 =
user
web site large-scale... I wouldn't but...) Some even say a coule of =
thousand
users would be the limit. I've posted similar queries about Zope's
scalibility on a number of occasions, but replies suggest "one might do
this" kind of stuff. There's been no concrete answer to these queries, =
an
answer out of real experience, not a guesstimation. That confuses me. =
Does
that mean nobody has reached the limit using Zope? 20GB of data is so =
normal
these days. I already have double the amount of data on my site. Guess =
I'll
start worrying about Zope's scalibility again.... Please convince me.
Anyone?

One mentioned months ago that Filestorage is so robust that it could
withstand most abuse I could throw at it. Okay, I believe it so. But the
question still remains. Why couldn't we have a FileStorage that can =
split
over partitions ( in multiple files, I mean, why one single golliath? ) =
and
that has an option to turn off versioning? One might say, "It's =
opensource,
please yourself. Write one yourself." ;-) But I just really wanted to =
know
why ZODB guys hasn't done that. Is there a reason I'm missing? Or is it
still on the to-do list?=20

Maybe I'm assuming wrong. Would you please elaborate what you mean by =
'RDBMS
backend'? Do you really mean I should write my own products to use MySQL =
as
backend bypassing ZODB , for example? Or is there an RDBMS storage ( =
Orcale
is not an option, if you mean Oracle storage )?

Thanks anyway for your comments.

Best Regards,
Wankyu Choi

-----Original Message-----
From: Andreas Jung [mailto:lists@andreas-jung.com]=20
Sent: Wednesday, January 29, 2003 2:55 PM
To: Wankyu Choi; zope@zope.org
Subject: Re: [Zope] Hardware for Zope + ZEO




--On Mittwoch, 29. Januar 2003 13:44 +0900 Wankyu Choi =
<wankyu@neoqst.com>=20
wrote:

>
>
> 3. I'm looking towards the Directory Storage over File Storate for=20
> tons of reasons, the reasons you might easily guess. I'll have six 73G =

> disks with RAID5, which means I'd have to let go 73G for storing=20
> parity information. That leaves me about 365G. At least 300G will be=20
> allocated for ZODB. Would FileStorage maintain its integrity if the db =

> grows to 300G? What I'm worried about most is that I can't make it=20
> versionless. Directory Storage has that option. Comments?

With 300GB you are at the point where I would think about a RDBMS as
backend. I assume maintaining or packing a 300GB Data.fs might be a pain
(20-30GB are already hard to handle).

-aj