[Grok-dev] Scalability of Grok web apps

[Jan-Wijbrand Kolman]

On 6/13/11 11:54 , Sebastian Ware wrote:
> Hi everybody!
> 
> We've built some pretty complex web sites using Grok and they perform
> really well compared to other stuff we've looked at. However I still
> feel I am lacking a bit in understanding what the scalability
> bottlenecks are and how to reduce the impact of these through
> design.
> 
> I would like to compile a list of potential bottlenecks and short
> explanations on why they might be a problem as a starting point to
> understand how to mitigate these issues.
> 
> Any input on the matter would be appreciated!

This is certainly a topic of interest to me.

Some potential bottlenecks that I know of:

* Session-usage for each request when using the SessionCredentials plugin.

* The former bullet is even much worse when Zope is serving static
resources. We saw a huge benefit from using fanstatic in this regard.

* Zope Page Templates. Maybe not so much a bottleneck, but we saw quite
a speedup by using Chameleon's page templates, esp. since it uses python
expressions by default.

* The catalog maybe too generically listening for object changes.

The directions we're considering for a current project (our former
projects were not very large scale, traffic-wise):

* RelStorage backend

* Using fanstatic, serving resources from a separate host

* Using beaker sessions and a re-implemention for SessionCredentials
that uses it (see also dolmen.beaker)

* Consider how to make updating the catalog only necessary for relevant
object changes. See for example ideas expressed here:
http://svn.zope.org/z3c.indexer/trunk/src/z3c/indexer/README.txt?rev=95906&view=markup

Let's indeed amend and discus this list.

regards, jw