[Zope] zope.org Members list

[Martijn Pieters]

From: "David Kankiewicz" <kankie@thegrid.net>
> > A next call to Members will return this cached information. Timeout of
the
> > cache currently is 60 minutes. So, the batched pages and the searches
use
> > this cached information.
>
> What about a ZCatalog being updated every hour??? I would think two or
> more hours would do...
> Have a cron job do it...

This might be a better idea. I'd use an external file, however.

Thtis would indeed solve quite a few points on the list I gave:

> > 1/ The cache has expired or is empty: we have to fill it again.

User will alway read from file, is fast (shared locking). A cache update
would do a exclusive lock at the end of the data gathering period, write
quickly, unlock again.

> > 2/ The cache is per-thread. <...>
> > 3/ The volatile attributes don't hang around as long as I expect them
to.
<...>

No longer.

> > 4/ The dictionary containing the cache may be coming from swapped
memory.
> > Currently, the Zope.org process grows quite large. This may have to do
with
> > the up to 42000 Folder objects loaded into memory at any time. The
Members
> > Folder holds references to 6000 Member Folders, and there can be up to 7
of
> > them (7 threads). This is an area where a BTree based folder would
_really_
> > help.
>
> should solve this too, unless people are surfing all those folders...

This doesn't solve this. It's not the surfing of all those folders, it's the
accessing of the Members Folder itself. It's that folder that holds the
references of 6000+ sub Folders.

<...>
> > 6/ CPU is hogged. The rendering of the ZQR is very expensive. If it
runs,
> > the whole of Zope.org suffers. I am seriously considering disabeling it.
> > People really onlly need the pre-rendered version. I was testing just
now,
> > had to wait way too long, and noticed someone was looking at both the
normal
> > ZQR (rendering from XML), and the print preview version. The Members
roster
> > came from cache.
>
> I know would come one of these days :)
> I'll be working on it tonight and starting to update it more actively.
> (more complete, different format, etc.)
> First thing I can think of is to make it all static, atleast the big
> ones.

Please. Amos was going to spend some time looking into optimizing the code,
but not offering a dynamic version would bemuch better. There really is no
need for one. Thanks!

Martijn Pieters
| Software Engineer    mailto:mj@digicool.com
| Digital Creations  http://www.digicool.com/
| Creators of Zope       http://www.zope.org/
|   The Open Source Web Application Server
---------------------------------------------