Hi, I've recently discovered zope and have the feeling that it may be the solution to a pending application issue. However, I've worked through the tutorial and I guess I'm no wiser than before, and it didn't really convince me either way; possibly leading me to believe it's aimed at small site solutions. Anyway, I have a project that will need to allow around 10,000 users (or less) to access at least a million documents. Each user is grouped into one of a small handful of access levels (determined by cookies). The documents are pre-existing PDFs, spreadsheet et al. I.e. they're all static. A solution along the lines of a standard cgi based bespoke application isn't particularly taxing from a programming point of view, but clearly wouldn't offer what I believe to be the potential of zope with it's prebuilt goodies. So, I'm wondering whether zope is up to the task (reliability and scalability), or whether I'm misunderstanding something. Any observations would be most welcome. Thanks, Tony
tonyl@vextech.net wrote:
Hi,
I've recently discovered zope and have the feeling that it may be the solution to a pending application issue. However, I've worked through the tutorial and I guess I'm no wiser than before, and it didn't really convince me either way; possibly leading me to believe it's aimed at small site solutions.
Not by intent! ;-)
Anyway, I have a project that will need to allow around 10,000 users (or less) to access at least a million documents. Each user is grouped into one of a small handful of access levels (determined by cookies). The documents are pre-existing PDFs, spreadsheet et al. I.e. they're all static.
Zope's access control is independent of the mechanism used to store state, so it can use cookies, basic auth, NT auth, etc. The hardest part about this is a fulltext searching requirement (if it exists). You may want to look at Kaivo's DocumentLibrary tool for this. If there is no fulltext searching requirement, then, well.. there are certainly a number of ways to do this in Zope. Some of them involve storing all the content in the ZODB. Others involve relational databases. Others involve the filesystem. It's hard to be more specific. ;-) But yes, it's certainly possible.
A solution along the lines of a standard cgi based bespoke application isn't particularly taxing from a programming point of view, but clearly wouldn't offer what I believe to be the potential of zope with it's prebuilt goodies.
Which goodies are you particularly interested in? Zope's got a lot of them, but different people are interested in different ones.
So, I'm wondering whether zope is up to the task (reliability and scalability), or whether I'm misunderstanding something.
A pretty good list of Zope sites is available via http://www.zopezen.org/links . Several sites on there are of the same scale as what you describe. -- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com """ Killing hundreds of birds with thousands of stones """
On Fri, Aug 31, 2001 at 03:09:19PM +0100, tonyl@vextech.net wrote:
Hi,
I've recently discovered zope and have the feeling that it may be the solution to a pending application issue. However, I've worked through the tutorial and I guess I'm no wiser than before, and it didn't really convince me either way; possibly leading me to believe it's aimed at small site solutions.
Anyway, I have a project that will need to allow around 10,000 users (or less) to access at least a million documents. Each user is grouped into one of a small handful of access levels (determined by cookies). The documents are pre-existing PDFs, spreadsheet et al. I.e. they're all static.
The big question in my mind is how to handle this number of users. Is each one to be authenticated? If so, my gut feeling is that that none of the standard userfolders will be much help.
A solution along the lines of a standard cgi based bespoke application isn't particularly taxing from a programming point of view, but clearly wouldn't offer what I believe to be the potential of zope with it's prebuilt goodies.
So, I'm wondering whether zope is up to the task (reliability and scalability), or whether I'm misunderstanding something.
scalability is probably not a worry. Reliabilty is good. To make it better, I run Jerome Alet's isdazope script. It checks to make sure zope is responsive at 1 minute intervals. If not, it restarts zope. I seem to get about 2 restarts per week. These usually involve write conflicts. Jim Penny
Any observations would be most welcome.
Thanks, Tony
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Jim Penny writes:
On Fri, Aug 31, 2001 at 03:09:19PM +0100, tonyl@vextech.net wrote: The big question in my mind is how to handle this number of users. Is each one to be authenticated? If so, my gut feeling is that that none of the standard userfolders will be much help. What do you mean with "standard userfolders"?
We use "GUF" (Generic User Folder), users are in an Oracle database. We currently have 2.000 users but the approach scales very well. I am not worried at all when we will have 10.000 users in 6 to 12 months. Per request we have three Oracle SQL queries, each usually served within 30 to 40 msec, almost independent from the number of users. I recently optimized GUF a bit to cache user objects. This saves about 0.1 sec per request. Dieter
Chris Withers writes:
Dieter Maurer wrote:
I recently optimized GUF a bit to cache user objects.
Is that in a released version of GUF? I started with a release version but did not release the modifications (it needs a Zope patch to work....).
Can GUF get users from an NT Domain Controller? Probably yes, if you work a bit harder, but there is NTUserFolder. Maybe, it's a better starting point.
Dieter
On Friday 31 Aug 2001 16:32 pm, Jim Penny wrote:
Reliabilty is good. To make it better, I run Jerome Alet's isdazope script. It checks to make sure zope is responsive at 1 minute intervals. If not, it restarts zope. I seem to get about 2 restarts per week. These usually involve write conflicts.
Are you saying zope crashes or fails twice a week? Surely that makes it unusable in a real environment, or is "write conflict" a bug in the site's implentation of zope? I'm confused!
On Fri, Aug 31, 2001 at 06:58:57PM +0100, tonyl wrote:
On Friday 31 Aug 2001 16:32 pm, Jim Penny wrote:
Reliabilty is good. To make it better, I run Jerome Alet's isdazope script. It checks to make sure zope is responsive at 1 minute intervals. If not, it restarts zope. I seem to get about 2 restarts per week. These usually involve write conflicts.
Are you saying zope crashes or fails twice a week? Surely that makes it unusable in a real environment, or is "write conflict" a bug in the site's implentation of zope? I'm confused!
(There is a known rare bug in the 2.3 series whereby the transaction machinery to an external database forgets to close transactions. I may be seeing by this. I may be taking an occasional cosmic ray, who knows.) Yes, I am saying that the version I have crashes once or twice a week. No, it does not make it at all unusable. There are 86400 minutes in a week. this makes it available 99.9976% of the time. It is automatically restarted after a minute without response goes by, so I do not have to be here, etc. Apache in my experience is better in terms of uptime. But I have had it crash, too. I have had PostgreSQL crash. I have has AS/400 services crash. Once you get beyond 3 9's of uptime, it starts to cost money. I am running on clone hardware with no ECC, etc. Prepare not to have 99.999% uptime, and you can handle 99.99% uptime quite well. BTW: I am not using zope in a "typical" profile. I am doing far more relational database work than appears to be normative. I am doing more concurrent writing than appears to be normative. (My PostgreSQL log file is about 15 Meg per day. so far today I have done 24,000 inserts, 1,000 inserts and 600 updates, and as the last day before holiday, this is very light). Given my experience, I can whole-heartedly recommend zope, and I suspect that most users would find less trouble with it than I do. isdazope is very useful. It can take care of restarting things so that a typical crash lasts no more than 60 seconds, unattended.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Friday 31 Aug 2001 20:18 pm, Jim Penny wrote:
Yes, I am saying that the version I have crashes once or twice a week. No, it does not make it at all unusable. There are 86400 minutes in a week. this makes it available 99.9976% of the time. It is automatically restarted after a minute without response goes by, so I do not have to be here, etc.
What planet are you using zope on Jim.? Your days last a real long time (or my maths is awful) ;-)
Apache in my experience is better in terms of uptime. But I have had it crash, too. I have had PostgreSQL crash. I have has AS/400 services crash. Once you get beyond 3 9's of uptime, it starts to cost money. I am running on clone hardware with no ECC, etc. Prepare not to have 99.999% uptime, and you can handle 99.99% uptime quite well.
For me, the issue is which of those minutes it's out of action. In my situation, the application will be hammered at the start of the month, week and morning (in that order). Most of the available minutes for statistical purposes won't have much in the way of system activity. So 1 minute in peak time is quite a high %age for the project. The equipment will probably be an 8 way sun box with oracle.
Given my experience, I can whole-heartedly recommend zope, and I suspect that most users would find less trouble with it than I do.
Sounds good to me (don't like the window for dhtml entry though)!
isdazope is very useful. It can take care of restarting things so that a typical crash lasts no more than 60 seconds, unattended.
I guess it has a user definable check-zope-is-up period? I'll soon change it if it hasn't! (if I can). Thanks, Tony
I think he did: 60 (seconds in a minute) * 60 (minutes in an hour) * 24 (hours in a day) = 86400 Tommy Innovation: The Best Way To Predict The Future ... Is To Create It.
What planet are you using zope on Jim.? Your days last a real long time (or my maths is awful) ;-)
Yes, I am saying that the version I have crashes once or twice a week. No, it does not make it at all unusable. There are 86400 minutes in a week. this makes it available 99.9976% of the time. It is automatically restarted after a minute without response goes by, so I do not have to be here, etc.
On Fri, Aug 31, 2001 at 08:39:29PM +0100, tonyl wrote:
On Friday 31 Aug 2001 20:18 pm, Jim Penny wrote:
Yes, I am saying that the version I have crashes once or twice a week. No, it does not make it at all unusable. There are 86400 minutes in a week. this makes it available 99.9976% of the time. It is automatically restarted after a minute without response goes by, so I do not have to be here, etc.
What planet are you using zope on Jim.? Your days last a real long time (or my maths is awful) ;-)
Ah damn, that is seconds per week. (Too used to thinking in seconds.) Make that 99.86% availablity.
Apache in my experience is better in terms of uptime. But I have had it crash, too. I have had PostgreSQL crash. I have has AS/400 services crash. Once you get beyond 3 9's of uptime, it starts to cost money. I am running on clone hardware with no ECC, etc. Prepare not to have 99.999% uptime, and you can handle 99.99% uptime quite well.
For me, the issue is which of those minutes it's out of action. In my situation, the application will be hammered at the start of the month, week and morning (in that order). Most of the available minutes for statistical purposes won't have much in the way of system activity. So 1 minute in peak time is quite a high %age for the project. The equipment will probably be an 8 way sun box with oracle.
Look, you are running on way more hardware with way better and way more expensive gear than I am used to. It sounds like you have an actual budget. If you can't get better uptime, I would be surprised. I come from a different situation entirely. In my situation, if it can't be done with hand-me-down hardware and a bit of sweat equity, then it can't be done. Also, all of that activity sounds like read activity. Read activity, in my experience, hardly ever triggers problems. at least 90% of my problems have been write related. (And I still would be a bit surprised if your user base is upset by an occaisional minute of downtime. The web has trained people way too well that downtime is normal!)
I guess it has a user definable check-zope-is-up period? I'll soon change it if it hasn't! (if I can).
It can be shortened, but beware (a bit). You do not want the period so short that zope cannot completely be reawakened. That is, you do not want to start hammering a restart process so hard that it starts taking down processes before they can start responding. Jim Penny
Thanks, Tony
On Friday, August 31, 2001, at 04:00 PM, Jim Penny wrote:
On Fri, Aug 31, 2001 at 08:39:29PM +0100, tonyl wrote:
On Friday 31 Aug 2001 20:18 pm, Jim Penny wrote:
Yes, I am saying that the version I have crashes once or twice a week. No, it does not make it at all unusable. There are 86400 minutes in a week. this makes it available 99.9976% of the time. It is automatically restarted after a minute without response goes by, so I do not have to be here, etc.
What planet are you using zope on Jim.? Your days last a real long time (or my maths is awful) ;-)
Ah damn, that is seconds per week. (Too used to thinking in seconds.) Make that 99.86% availablity.
No, that's still seconds per day :^) 60 seconds * 60 minutes * 24 hours * 7 days = 604800 seconds per week 2 minutes = 120 seconds which is 0.0001984127 of the week, or 99.98...% uptime. Still not bad. me = { 'name' : 'Zachery Bir', 'email' : 'zbir@urbanape.com', 'voice' : '(804) 353-3742', 'url' : 'http://www.urbanape.com/' }
On Fri, 2001-08-31 at 13:39, tonyl wrote: Sorry, coming in a bit late in the game perhaps but ...
On Friday 31 Aug 2001 20:18 pm, Jim Penny wrote:
Yes, I am saying that the version I have crashes once or twice a week. No, it does not make it at all unusable. There are 86400 minutes in a week. this makes it available 99.9976% of the time. It is automatically restarted after a minute without response goes by, so I do not have to be here, etc.
What planet are you using zope on Jim.? Your days last a real long time (or my maths is awful) ;-)
Apache in my experience is better in terms of uptime. But I have had it crash, too. I have had PostgreSQL crash. I have has AS/400 services crash. Once you get beyond 3 9's of uptime, it starts to cost money. I am running on clone hardware with no ECC, etc. Prepare not to have 99.999% uptime, and you can handle 99.99% uptime quite well.
For me, the issue is which of those minutes it's out of action. In my situation, the application will be hammered at the start of the month, week and morning (in that order). Most of the available minutes for statistical purposes won't have much in the way of system activity. So 1 minute in peak time is quite a high %age for the project. The equipment will probably be an 8 way sun box with oracle.
In your case, I would recommend ZEO. ZEO will take better advantage of your hardware, and will provide much more stability, or at least redundancy. I would run, say, 4-8 ZEO Servers, serving from a local ZSS. I would then put these behind, say, Balance, for example. You could further enhance things, by perhaps putting the whole sha! behind Apache, using Proxypass, or behind Squid. This will have the added benefit of caching your static data, which will, in turn, free up Zope a little more. I have done similar setups on HP N-Class machines. It works beautifully.
Given my experience, I can whole-heartedly recommend zope, and I suspect that most users would find less trouble with it than I do.
Sounds good to me (don't like the window for dhtml entry though)!
So don't use it. I use gvim/vim for my editing, and upload using the upload button. :) Cheers, Bill
On Fri, Aug 31, 2001 at 03:18:07PM -0400, Jim Penny wrote:
BTW: I am not using zope in a "typical" profile. I am doing far more relational database work than appears to be normative. I am doing more concurrent writing than appears to be normative. (My PostgreSQL log file is about 15 Meg per day. so far today I have done 24,000 inserts, 1,000 inserts and 600 updates, and as the last day before holiday, this is very light).
We've just set up a Zope student portal for a university faculty here in Bergen http://realfag.uib.no/. It's pretty new, and we've still got a few minor bugs to sort out, but I think it's starting to look quite good (10-15.000 hits a day so far, and we just went public a week ago). We're using postgres and popy, running on a dual cpu Sun Blade 1000. There's been a few crashes hopefully caused by an old PoPy/ZPoPyDA version, but I upgraded to the latest version today. Maybe you could give some advise on how to optimally do database connections. So far we're only using 4 database connections, where it looked naturally at the time. At times we see maybe 50-100 database transactions a second (mostly reads). Are there any point in splitting the load over more database connections? What about the NUMBER_OF_THREADS in z2? I set this at 80, but that's mostly a wild guess from my side. Any advise on how many threads to use for a high traffic site doing both ZODB3 and postgres/popy? Any advise, or pointers to information on how to configure zope+postgres+apache for optimal performance on a dual cpu machine would be greatly appreciated. -jf
Jan-Frode Myklebust writes:
Maybe you could give some advise on how to optimally do database connections. So far we're only using 4 database connections, where it looked naturally at the time. At times we see maybe 50-100 database transactions a second (mostly reads). Are there any point in splitting the load over more database connections? What about the NUMBER_OF_THREADS in z2? I set this at 80, but that's mostly a wild guess from my side. Any advise on how many threads to use for a high traffic site doing both ZODB3 and postgres/popy? Without changing Zope's code, there is an upper limit on the number of concurrent connections to the ZODB of 7. With this limit, it is not effective to set the number of threads to a significant larger value.
Dieter
tonyl writes:
Are you saying zope crashes or fails twice a week? Surely that makes it unusable in a real environment, or is "write conflict" a bug in the site's implentation of zope? I'm confused! Usually, Zope does not crash when a "write conflict" occurs....
Our Zope installation is much more stable.... It usually only gets into problems when the Oracle database behind it plays havoc or someone programmed an "infinite loop". Dieter
On Sat, Sep 01, 2001 at 10:47:13PM +0200, Dieter Maurer wrote:
tonyl writes:
Are you saying zope crashes or fails twice a week? Surely that makes it unusable in a real environment, or is "write conflict" a bug in the site's implentation of zope? I'm confused! Usually, Zope does not crash when a "write conflict" occurs....
Our Zope installation is much more stable....
It usually only gets into problems when the Oracle database behind it plays havoc or someone programmed an "infinite loop".
Usually it does not crash for me either; I would guess actual all-out, core dump type crashes occur in no more than 10% of restarts. Usually zope just goes catatonic for an unpredictable amount of time. That is, left to its own devices, it will normally recover, but in my experience, it is sometimes very sluggish after a conflict (but not always). So, I find that leaving it to its own devices results in telephone calls. Using a monitor like isdazope to try to get an actual response from the server, and restarting when it is non-responsive, has cut this kind of call down to essentially nothing. Now most calls are do to me doing brain surgery on an active site. I still get telephone calls, but it is clearly my fault, and I feel better having problems that I know I caused myself rather than problems that I can't quite put a finger on. Jim Penny
Dieter
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
participants (10)
-
Bill Anderson -
Chris McDonough -
Chris Withers -
Dieter Maurer -
Jan-Frode Myklebust -
Jim Penny -
Tommy Johnson -
tonyl -
tonyl＠vextech.net
-
Zachery Bir