[Grok-dev] horizontal scaling

[Christopher Mulcahy]

Scaling horizontally. 

I am currently using my own home-brewed python web-application framework, 
but I would like to migrate to a better option.  I have always admired Zope 
from afar ( very far ) but I am very intrigued by Zope3 and ( by extension ) 
Grok.   I bought Philipp and Stephan's books and I have been trying to 
determine if Zope3 technologies are what I am looking for. 

I have discarded the other python "Rails-alike" frameworks because as far as 
I can tell, they ( and Rails ) suffer from the same problem.   Every 
web-application ( no matter how small or large ) requires its own 
long-running process or processes.   The larger an application becomes, the 
more functionality it acquires, the larger its resource foot-print becomes 
even if it is not handling any requests.   This does not scale. 

I wrote my own python framework because although I loathe PHP, I recognized 
that it had some undeniable strengths.   I own a small web-hosting company, 
and I buy, install,  and administer the machines my applications run on, so 
I have a very SysAdmin ( as opposed to Web-Developer ) way of looking at 
things. 

PHP's strength?   Horizontal Scaling ( for lack of a better word ). 

A web-server might have 200-odd virtual-hosts serving 200 different crappy 
PHP bulletin-boards, dodgy shopping carts, and bug-ridden CMS applications.  
The run-time costs of hosting these 200 different applications in 200 
different Apache virtual hosts at 2:30 AM when the requests dribble in at 
1/second or so?   Next to nothing.   The resource bloat of mod_php in all 
thoses httpds is neglible ( the dodgy code bloat is something else. )    
Under load, the "cost" of hosting all these disparate apps with 3 to 300 php 
scripts each, is simply the runtime cost of each script request.    It 

Note, I am not advocating for PHP.  Its weaknesses are too numerous to 
mention, and as a matter of fact I don't host hundreds of crappy PHP 
bulletin-boards, blogs, and CMS apps but it is for reasons of security, the 
prospect of painful unpaid code-maintainence, and general aesthetics, not 
resource utilization.   All I am saying is that it is possible to host 
hundreds of websites each with hundreds of possible dynamic URLs with 
processor and memory to spare on 4 year-old hardware because the general 
"machinery" for processing all these requests is  ready and running ( I 
don't want to say "in-process" because I don't believe in the fat buggy 
httpds brought on by the whole mod_perl, mod_python thing and "external 
process" techniques like SCGI work very well for me ) and GENERIC, each 
mod_php enabled httpd, or php fast_cgi process capable of serving one 
"application" as well as another from request to request. 

Zope3 is very large and I haven't been able to get the 30,000 foot view yet. 

Most simply I suppose:
Is it possible for 1 zope3 application server ( or a family of zope3 
application servers sharing a ZEO or relstorage-like back-end ) to provide 
multiple Grok application instances for many ( maybe not 200, but 30 or 40 ) 
different virtual hosts ( fronted by squid, apache et.al ) . 

Can an application that has not been accessed in hours ( or days ) be 
"passivated" and its runtime resource costs be reduced to disk-space and a 
tiny entry in an in-memory hashtable? 

I know that zope3  ( and thereby Grok ) is a lot heavier than my own 
framework, but it brings a lot more to the table.
I just can't afford to run one for every site ( large or small ) that needs 
dynamic content. 

I have a vision of  one redundant zope3-cluster ( backed by relstorage on a 
slony-replicated postgresql cluster ) that can provide all of the 
web-application infrastructure for all of my development work for various 
disparate customers going forward. 

If this is feasible? 

If this were possible it would be a slam-dunk for the adoption of grok/zope3 
over rails, turbogears, django, et.al. 

Regards, 

Christopher  Mulcahy