[Zope-dev] How to make Zope fail nicely under high load?

[Erik A.Dahl]

using the _v_ variables won't work for this since different threads 
won't agree on who was latest.  Its definitely a bad idea to do queries 
for these three data points.  What about putting them in the 
temp_folder then you can still use ZEO with a bunch of clients that 
will all share the same data but you don't need to write the values to 
disk every time they change.  Even putting them in a normal ZODB 
storage would be a _lot_ faster than doing a query.

-EAD

On Feb 12, 2004, at 8:19 AM, Bjorn Stabell wrote:

> Toby wrote:
>>> One of the optimization we're thinking of is storing results of
>>> ZCatalog searches (e.g., number of replies to postings) in volatile
>>> variables so we don't have to run the catalog search at all.  We'd
>>> like to use memory space shared between threads for this.
>>> Using ZEO would require us to store this in the ZODB.
>>
>> Storing that in ZODB would be a bad idea. Theres no reason to
>> think that this cache would be faster than ZCatalog.
>>
>> I dont see why you would be *required* to store in ZODB...
>> just dont share your cache between publisher threads on different
>> Zope instances.
>>
>> (my apologies if this is obvious)
>
> Okay, for example, since it's a BBS we need to have quick access to:
>
> 	the latest poster
> 	the latest posting
> 	the latest registered member
>
> All of these are complex queries (in the case of the latest registered
> member it's not even a query but a brute-force search).  What I think
> most ASP/PHP boards do is that they store these values in the database
> instead of querying for them all the time.
>
> Since many of these can be found through queries, they don't really 
> need
> persistent storage, but updates need to be seen by all threads; thus 
> the
> requirement for ZODB if we use ZEO, but the possibility to use (even
> faster) shared memory if we don't use ZEO.
>
> In effect we want to do something like this:
>
> def getLatestPoster(self):
> 	if not hasattr(self, '_v_latestPoster'):
> 		self._v_latestPoster = whatever_query()
> 	return self._v_latestPoster
> 		return self._v_latestPoster
>
> def setLatestPoster(self, poster):
> 	self._v_latestPoster = poster
>
> But instead of having thread-local variables we wanted to use Dieter
> Maurer's SharedResource in order to share the cache between threads:
>
> http://www.dieter.handshake.de/pyprojects/zope/SharedResource.html
>
> Regards,
> -- 
> Bjorn
>
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope )