[Zope-dev] How to make Zope fail nicely under high load?

[Bjorn Stabell]

Toby wrote:
> > One of the optimization we're thinking of is storing results of 
> > ZCatalog searches (e.g., number of replies to postings) in volatile 
> > variables so we don't have to run the catalog search at all.  We'd 
> > like to use memory space shared between threads for this.  
> > Using ZEO would require us to store this in the ZODB.
> 
> Storing that in ZODB would be a bad idea. Theres no reason to 
> think that this cache would be faster than ZCatalog.
>
> I dont see why you would be *required* to store in ZODB... 
> just dont share your cache between publisher threads on different
> Zope instances.
> 
> (my apologies if this is obvious)

Okay, for example, since it's a BBS we need to have quick access to:

	the latest poster
	the latest posting
	the latest registered member

All of these are complex queries (in the case of the latest registered
member it's not even a query but a brute-force search).  What I think
most ASP/PHP boards do is that they store these values in the database
instead of querying for them all the time.

Since many of these can be found through queries, they don't really need
persistent storage, but updates need to be seen by all threads; thus the
requirement for ZODB if we use ZEO, but the possibility to use (even
faster) shared memory if we don't use ZEO.

In effect we want to do something like this:

def getLatestPoster(self):
	if not hasattr(self, '_v_latestPoster'):
		self._v_latestPoster = whatever_query()
	return self._v_latestPoster
		return self._v_latestPoster

def setLatestPoster(self, poster):
	self._v_latestPoster = poster

But instead of having thread-local variables we wanted to use Dieter
Maurer's SharedResource in order to share the cache between threads:

http://www.dieter.handshake.de/pyprojects/zope/SharedResource.html

Regards,
-- 
Bjorn