-----Original Message----- From: Jimmie Houchin [mailto:jhouchin@texoma.net] Sent: Monday, June 14, 1999 2:41 PM To: zope@zope.org Subject: [Zope] Large Portal requirements (was: Other web servers?)

nto.

Howabout for database intensive situations like one encounters in some of the larger sites like Altavista, Goto, Yahoo, Amazon, etc.

Here are very, very large databases, probably very large indexes, multi-millions of objects, multi-multi-gigabytes to search, index. My concern is because Python does not scale with multiple processor machines, this limits ones ability to throw hardware at the situation since you are limited to the fastest single processor.

In the future, we will be have the client server model of ZODB3 (which will be a commercial product) that lets you have multiple Zope frontend processes (possibly on different machines) accessing multiple database backend processes (possibly on different machines). Also, ZServer is currently multi-threaded, but there is no theoretical limitation to making it multi-process, in which each Zope instance will have it's own Python and thus it's own global Python lock, thus giving the OS the ability to schedule multiple processes among multiple processors, a little heavier than threads on multi-processor, but *vastly* easier to impliment and far more stable.

Do I misunderstand anything.

Would Zope be appropriate for these situations? How well would Zope handle it?

For example in an article I read concerning the delay in Intel bringing out it's chipset for 8way Pentium III servers. Barnes and Noble was talking about beta-testing an 8way server to speed up their searches.

'Searching' is a whole nother question. If n Zope *processes* are all doing a search through the index at the same time and you have n processors, then each process will complete the operation without blocking each other, if multiple threads do it, then they will block each other. However, multiple processors will not speed up the time it takes to do *one* search, it will just allow multiple searches to be done concurently (which in high load situations makes each individual search look much faster). Getting into the ability break a searching algorithm down into multiple processors is complex and probably only pays off when your talking about searching through *massive* quantities of data. I mean big. If you had a Zope index with one million entries in it, you would, worst case, have to go through 20 comparison operations to find the element your looking for (Zope indexing uses a very efficient binary tree structure to store it's indexes).

Now I understand that different development models yield different results and I don't know how Barnes and Noble's website is built. I do know that B&N has a very large database with multi-millions of objects with many, many gigs of data.

Would Zope be appropriate?

Sure, I can see Zope easily scaling to this level.

I know that my website will have similar and greater requirements concerning database size and searching. Hopefully hits/traffics also will be high.

Zope and Python are my first choice.

Cool! You won't be disapointed. -Michel

Thanks for any help.

Jimmie Houchin

[snip]

_______________________________________________ Zope maillist - Zope@zope.org http://www.zope.org/mailman/listinfo/zope

(For developer-specific issues, use the companion list, zope-dev@zope.org - http://www.zope.org/mailman/listinfo/zope-dev )