How big a site can Zope handle?
Hi I'm a newbie with Zope, but have used it to build a fairly complex site recently. This site only had to support 91 users, all with access permissions, and quite a low traffic level. My studio is pitching to design a build a website for a very large organisation and I think Zope would be perfect. I need to know that it will be able to handle the amount of users and hits that this site will have to deal with. About 275,000 users to have accounts and about 150,000 hits a day (at the moment but set to grow quickly). Will Zope be able to deal with this? Cheers Matt ------------ Matt Carey Text Matters 37 Upper Redlands Road, Information design: Reading RG1 5JE, UK We help explain things using phone +44 (0)118 986 8313 .language fax +44 (0)118 931 3743 .design email matt@textmatters.com .systems web http://www.textmatters.com .process
I need to know that it will be able to handle the amount of users and hits that this site will have to deal with. About 275,000 users to have accounts and about 150,000 hits a day (at the moment but set to grow quickly).
Easily. You might want to look at alternative user folder like exUserFolder to handle that many users. Things like ZEO and the all powerful RAM Cache Zope can easily handle that load on a commodity box. Once it gets really huge you'll want to start looking at things like Squid and other Cachers. Cheers. -- Andy McKay. ----- Original Message ----- From: "matt carey" <matt@textmatters.com> To: <zope@zope.org> Sent: Thursday, November 29, 2001 9:08 AM Subject: [Zope] How big a site can Zope handle?
Hi
I'm a newbie with Zope, but have used it to build a fairly complex site recently. This site only had to support 91 users, all with access permissions, and quite a low traffic level.
My studio is pitching to design a build a website for a very large organisation and I think Zope would be perfect.
I need to know that it will be able to handle the amount of users and hits that this site will have to deal with. About 275,000 users to have accounts and about 150,000 hits a day (at the moment but set to grow quickly).
Will Zope be able to deal with this?
Cheers
Matt
------------ Matt Carey Text Matters 37 Upper Redlands Road, Information design: Reading RG1 5JE, UK We help explain things using phone +44 (0)118 986 8313 .language fax +44 (0)118 931 3743 .design email matt@textmatters.com .systems web http://www.textmatters.com .process
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
From: "Andy McKay" <andym@ActiveState.com>
I need to know that it will be able to handle the amount of users and hits that this site will have to deal with. About 275,000 users to have accounts and about 150,000 hits a day (at the moment but set to grow quickly).
Easily.
How much of a difference does it make if most of those hits make a transaction (ie change something in the database). Hmm. That was a very loosely put question, maybe I should be more specific so I'll get a better answer. :-) If I have a nice big commodity box, say a 1GHz PC, and have users that with all of their hits make some kind of transaction, say, writing in a discussion forum, or reading in a discussion forum (and thereby marking the posts as read, hence the transaction). How many users would such a system be able to handle? 100, 1000, 10.000, 100.000? Any experiences on this?
This is a highly subjective question and depends entirely on how the software that does the discussion system is written and what persistence mechanism it uses. There is no general answer. The answer (and I've found this out the hard way) is to do testing of your software before you put it in to production. Zope as a platform can support high transaction rates. The rest is up to you. - C ----- Original Message ----- From: "Lennart Regebro" <lennart@regebro.nu> To: <zope@zope.org> Sent: Thursday, November 29, 2001 3:20 PM Subject: Re: [Zope] How big a site can Zope handle?
From: "Andy McKay" <andym@ActiveState.com>
I need to know that it will be able to handle the amount of users and hits that this site will have to deal with. About 275,000 users to have accounts and about 150,000 hits a day (at the moment but set to grow quickly).
Easily.
How much of a difference does it make if most of those hits make a transaction (ie change something in the database).
Hmm. That was a very loosely put question, maybe I should be more specific so I'll get a better answer. :-)
If I have a nice big commodity box, say a 1GHz PC, and have users that with all of their hits make some kind of transaction, say, writing in a discussion forum, or reading in a discussion forum (and thereby marking the posts as read, hence the transaction). How many users would such a system be able to handle? 100, 1000, 10.000, 100.000? Any experiences on this?
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
From: "Chris McDonough" <chrism@zope.com>
This is a highly subjective question and depends entirely on how the software that does the discussion system is written and what persistence mechanism it uses. There is no general answer. The answer (and I've found this out the hard way) is to do testing of your software before you put it in to production. Zope as a platform can support high transaction rates. The rest is up to you.
So, the question is really then: "How do I make my software support a large amount of concurrent posts?" :-) I'm not 100% sure what you mean with "persistence mechanisms". I was first and foremost thinking about just storing things in ZODB, and not using an external DB if that's what you are referring too.
So, the question is really then: "How do I make my software support a large amount of concurrent posts?" :-)
Probably by using a relational database. The ZODB is not real good at concurrent writes, although it excels at reads.
I'm not 100% sure what you mean with "persistence mechanisms". I was first and foremost thinking about just storing things in ZODB, and not using an external DB if that's what you are referring too.
That's what I meant. - C
How much of a difference does it make if most of those hits make a transaction (ie change something in the database).
It depends, Zope is not designed for lots of changes, you'll want to look at an external RDBMs. The limiting factor is likely to be your database here.
If I have a nice big commodity box, say a 1GHz PC, and have users that with all of their hits make some kind of transaction, say, writing in a discussion forum, or reading in a discussion forum (and thereby marking the posts as read, hence the transaction). How many users would such a system be able to handle? 100, 1000, 10.000, 100.000? Any experiences on this?
It depends on so many variables its almost impossible to say. Again its the DB that will decide not Zope. Cheers. -- Andy McKay. ----- Original Message ----- From: "Lennart Regebro" <lennart@regebro.nu> To: <zope@zope.org> Sent: Thursday, November 29, 2001 12:20 PM Subject: Re: [Zope] How big a site can Zope handle?
From: "Andy McKay" <andym@ActiveState.com>
I need to know that it will be able to handle the amount of users and hits that this site will have to deal with. About 275,000 users to have accounts and about 150,000 hits a day (at the moment but set to grow quickly).
Easily.
How much of a difference does it make if most of those hits make a transaction (ie change something in the database).
Hmm. That was a very loosely put question, maybe I should be more specific so I'll get a better answer. :-)
If I have a nice big commodity box, say a 1GHz PC, and have users that with all of their hits make some kind of transaction, say, writing in a discussion forum, or reading in a discussion forum (and thereby marking the posts as read, hence the transaction). How many users would such a system be able to handle? 100, 1000, 10.000, 100.000? Any experiences on this?
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Thu, 29 Nov 2001 21:20:58 +0100, "Lennart Regebro" <lennart@regebro.nu> wrote:
If I have a nice big commodity box, say a 1GHz PC, and have users that with all of their hits make some kind of transaction, say,
writing in a discussion forum,
If usage of your forums is anything like way I read the zope mailing list, then less than 1% of transactions will be posting new articles.
or reading in a discussion forum (and thereby marking the posts as read, hence the transaction).
Other posters have suggested that zope is not good for high-write throughput. Thats generally true, and you might want to consider using a different way of storing the read/unread status. Cookies?
How many users would such a system be able to handle? 100, 1000, 10.000, 100.000? Any experiences on this?
A hand-waving ballpark estimate assuming you dont store read/unread status in the ODB: 10 to 100 read transactions per second (the variation depends on your page complexity). Even better than that, you can use a caching front-end proxy to multiply that up. 2 to 20 impure-read transactions if you do. and you cant cache :-( I hope that helps. Toby Dickenson tdickenson@geminidataloggers.com
From: "Toby Dickenson" <tdickenson@devmail.geminidataloggers.co.uk>
If usage of your forums is anything like way I read the zope mailing list, then less than 1% of transactions will be posting new articles.
Sure, but the remainding 99% will set a read-flag. We have some ideas of how to merge loads of these read-flags into one transaction, but I'm not sure it will be useful. I'm just very lazy and don't want to implement something I don't need. :-)
Other posters have suggested that zope is not good for high-write throughput. Thats generally true, and you might want to consider using a different way of storing the read/unread status. Cookies?
Nah, it has to be set on a user level, not on a computer level.
A hand-waving ballpark estimate assuming you dont store read/unread status in the ODB: 10 to 100 read transactions per second (the variation depends on your page complexity). Even better than that, you can use a caching front-end proxy to multiply that up.
2 to 20 impure-read transactions if you do. and you cant cache :-(
I hope that helps.
Very helpful, thanks!
Matt Carey Text Matters 37 Upper Redlands Road, Information design: Reading RG1 5JE, UK We help explain things using
Hi Matt, Just noticed that you work up the road from where I live :-) I'm in Tilehurst, NIP is in Crowthorne down near Bracknell. Pop in for a beer some time :-) Chris
Chris Withers wrote:
Pop in for a beer some time :-)
List CC not entirely intentional, nevermind, if anyone else is near, I got plenty of beer :-) (and if anyone wants to shoot me for something I should have done, I don't really live in Tilehurst, honest...) *grinz* Chris
participants (6)
-
Andy McKay -
Chris McDonough -
Chris Withers -
Lennart Regebro -
matt carey -
Toby Dickenson