September 2003 - Zope-Dev

RE: [Zope-dev] AW: Caching prob with AHCM and headers
by Bjorn Stabell 13 Sep '03

13 Sep '03

Carsten wrote: > Well, sadly I didn't find the time to pursue the issue any > further yet. We're kneedeep in work to get the system up and > running and caching isn't top priority so far. I don't even > know if someone patched the AHCM yet, simply haven't looked. > > But as someone suggested somewhere and you can see in > http://mail.zope.org/pipermail/zope/2003-April/135101.html it > works if you set the headers yourself. So if you need caching > _now_ and AHCM is not fixed yet, change the object used so > that it sets the headers correctly itself. > > I know that this solution is ikky but I don't know when I > will have time to tackle the issue of a comfortable cache > system. Though I know I will have to solve it sometime :) I know, I was just trying to figure out how this was supposed to work. Currently setting Last-Modified seems to be the only way to get something cached in Apache; Expires alone is no good, and adding an Etag header doesn't seem to have any effect either. Maybe this should be classified as an Apache problem? Shouldn't Apache cache pages that have Expires? Setting Last-Modified to the current time isn't always the right thing; some objects, .e.g., Files, Images, Documents, have last modification times, and using those would be better. Not sure if the accelerator can be smart enough to know about these, though. Bye, -- Bjorn

4 3

Zope.org maintenance update afternoon EST
by Brian Lloyd 12 Sep '03

12 Sep '03

Hi all - I will be doing some maintenance on the zope.org database, packing it and performing a reindex to get rid of some duplicate search results that we've been seeing. There should be little to no interruption of service, but the pack / reindex / replace will take an hour or so. Please avoid adding new content to the site for this afternoon (EST), as I may need to take a copy of the db to do these operations in order to avoid impacting performance on the main site. thanks, Brian Lloyd brian(a)zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com

1 0

Catalog performance
by Nguyen Quan Son 12 Sep '03

12 Sep '03

Hi, I have a problem with performance and memory consumption when trying to do some statistics, using following code: ... docs = container.portal_catalog(meta_type='Document', ...) for doc in docs: obj = doc.getObject() value = obj.attr ... With about 10.000 documents this Python script takes 10 minutes and more than 500MB of memory, after that I had to restart Zope. I am running Zope 2.6.1 + Plone 1.0 on Windows 2000, Xeon P4 with 1GB RAM. What's wrong with this code? Any suggestion is appreciated. Nguyen Quan Son.

8 12

Re: [Zope-dev] PythonLibraries Product
by Tres Seaver 12 Sep '03

12 Sep '03

Evan Simpson wrote: > Seb Bacon wrote: > > Could you provide a brief summary of why this is better than a > > folder of python scripts? Perhaps a use case which illustrates the > > problems of the current way of doing things? > > It isn't *better* than a Folder of Scripts, it's *different* :-) I > have two weak use-cases, a longstanding "I'm going to write that some > day" itch, and some aesthetic arguments. > > One of the more frequently stated complaints about ZPTs is that to > properly remove non-presentation logic and ugly little Python > expressions from a template, you either need a pile of little Scripts, > or a single "controller" Script that precalculates everything and > passes it to the template. I've settled, fairly happily, on the > second solution. > > Both of these solutions suffer from the "two or more objects to > accomplish one task" management problem. I usually combine the > objects in a dedicated Folder, which is still a bit awkward, and > doesn't solve everything. One drawback is that the individual > components are all published, despite the fact that I only intend > one of them to be web-addressable. It would be very handy to attach > the Scripts directly to the template, but the straightforward way of > doing that means making the template a container, and opens a whole > can of worms. Instead, what if templates had a Library tab, and it > worked with External Editor? It would actually be easier to *just* > write this, and not have independent Library objects, but generalizing > this capability feels right to me Actually, the restricted case is the one which has the real win; the "free-floating" library is pretty, but not semanticaally needed. An added argument: a ZPT with its own private library becomes, in effect, a Zope3 view component; adopting such beasts will ease migration to Zope3. For the filesystem representation: what if we just have two files for templates with libraries: 'foo.html' and 'foo.html.py'? Tools will like that better than either of the approaches which try to preserve the twins as a single file: - Embedding the Python code into an HTML/XML comment at the head or foot of the template in the .pt file will not let editors help with syntax highlighting. - In addition to messing up syntax highlighting, embedding the template as a triple-quoted string literal in the .py file feels *really* nasty (almost as nasty as Zope3's <script> tag). And there is yet another argument for the bound library: it shows how unnecessary, as well has how evil, that <script> tag hack is. Refactoring a "one-off" template-cum-library will be simple, too: once the refactored code is moved to a "real" module (even a persistent module in Zope3!), the library can just contain 'from realmodule import usefulFunction', and the template won't need to change. I would vote for having the bound library's names inserted into the 'template' TALES namespace, BTW. Tres. -- =============================================================== Tres Seaver tseaver(a)zope.com Zope Corporation "Zope Dealers" http://www.zope.com

5 4

PythonLibraries Product
by Evan Simpson 12 Sep '03

12 Sep '03

I'm thinking seriously about writing a Product to provide collections of Python functions defined by a single source text -- PythonLibraries. This would *not* be the same as Zope 3's persistent modules, although it would provide some of the same benefits. Here's the README.txt: Python Libraries The Python Libraries Product provides support for collections of restricted Python code. A Python Library is similar to a Folder full of Python-based Scripts, except that the functions in the Library are more like ordinary Python functions than Scripts, and a single persistent global variable namespace is shared among the functions in the Library. A Library obeys the same security restrictions, and has access to the same set of modules, as a Script. Libraries are *not* very much like Python modules or packages, except in that they both are convenient places to keep groups of functions. You cannot import a Library, define Python classes in one, or use the 'print' statement outside of function definitions. When changes to a Library are saved, its source code is executed. This process, known as "Library initialization", also occurs whenever the Library is loaded into memory from the ZODB. Global variables defined during initialization are divided into three groups, depending on the kind of value to which they refer: functions, simple data, and modules. There is one additional global variable, named 'Library', that is defined both during initialization and function evaluation, and which cannot be deleted or rebound. It refers to the Library object itself. Functions are made available to other Zope code as Library attributes. A function 'do_it' contained in the Library located at '/foo/myLib' may be called by a TALES expression such as "here/foo/myLib/do_it" or the Python expression "context.foo.myLib.do_it()". Function names that conflict with methods of the Library object are syntax errors. A Library's functions are not published by default, which means that they cannot be accessed by URL through ZPublisher. There is a Library method 'pl_publish()' that can be used to explicitly publish a function. Simple data includes Python built-in types such as numbers, strings, lists, and dictionaries. Variables with simple data are persistent, and their value is retained even when the Library is changed or reloaded, although the execution of the code can remove or overwrite them. These variables are not visible to other Zope objects. Modules are imported Python modules. These are not visible to other Zope objects, and do not persist -- they are re-imported each time the Library is changed or loaded. Unlike Scripts, Libraries do not have a Bindings tab. In order to access context objects, such as the Library's container, the root Zope object, or the authenticated user, functions must use a global variable created by a call to the 'pl_bind()' method of the Library. This method takes two arguments: a variable name, and a TALES expression. The TALES expression will be evaluated the first time that the variable is used in each call to a Library function, and the value will be cached for subsequent uses within the same call. For example, examine the following snippet of Library code: Library.pl_bind('user', 'user') Library.pl_bind('thingy', 'here/thingy | nothing') def f(): if user.has_role('Authenticated') and thingy is not None: print user, thingy return printed When the function 'f' is called, the 'user' and 'thingy' variables will be evaluated in its first line, and the values will be reused in the second line, if the condition is true. If 'f' is called again, in the same request or not, the bound variables will be re-evaluated. Bound variables are not available during Library initialization, since they are unlikely to evaluate meaningfully when a Library is loaded from the ZODB.

4 5

Zope HEAD test.py --dir doesn't work...
by Chris Withers 11 Sep '03

11 Sep '03

...can anyone else confirm this? It doesn't appear to ever find any tests. How should it be used? (just to check I'm doing the right thing...) cheers, Chris PS: CC'ing Jeremy in since he's the last person to touch test.py...

1 0

Re: [Zope-dev] PythonLibraries Product
by Tres Seaver 11 Sep '03

11 Sep '03

On Wed, 2003-09-10 at 23:46, Evan Simpson wrote: > Tres Seaver wrote: > > Actually, the restricted case is the one which has the real win; the > > "free-floating" library is pretty, but not semanticaally needed. An > > added argument: a ZPT with its own private library becomes, in effect, > > a Zope3 view component; adopting such beasts will ease migration to > > Zope3. > > Agreed. > > > For the filesystem representation: what if we just have two files for > > templates with libraries: 'foo.html' and 'foo.html.py'? > > Eh. I think filesytem representations should handle this sort of thing > much more generally. Assuming that the Zope object's ID is 'foo.html', > there should be a text file of that name (encoded with quote_plus, which > makes no difference in this case), and a directory named 'foo.html,'. > Inside of this directory, there would be a 'library.py' (in this > particular case), along with files such as 'properties' and 'proxies'. Hmm, I'm thinking about this for the CMF. In CMF 1.3.x, a template which has a cache manager, or a title, puts those values in a simple 'name=value'-formatted file, with extension '.properties'. Permission mappings go in a separate file, with extension '.security'. So, the filesystem-based template 'foo_view' might have as many as three files associated with it: - 'foo_view.pt' - 'foo_view.pt.properties' - 'foo_view.pt.security' CMF 1.4 unified the two files into a single, '.ini'-formatted one: - 'foo_view.pt' - 'foo_view.pt.metadata' It feels more natural to me just to add another file: - 'foo_view.pt.py' This is especially nice for many CMF skin templates (those which contain forms), because I can probably *remove* the separate template which serves as the "POST-target" for the form! Instead, such templates will post *to themselves*, and use the bound library to do the validation / updates. In this case, we go from: - 'foo_edit_form.pt' - 'foo_edit.py' to: - 'foo_edit_form.pt' - 'foo_edit_form.pt.py' > > > And there is yet another argument for the bound library: it shows how > > unnecessary, as well has how evil, that <script> tag hack is. > > Yep :-) > > > > Refactoring a "one-off" template-cum-library will be simple, too: > > Cool! > > > I would vote for having the bound library's names inserted into the > > 'template' TALES namespace, BTW. > > Tricky -- right now, that's just the template object itself. I'd just > introduce a builtin 'library' variable, and probably dump the function > names into the global namespace as well. +1, then. Tres. -- =============================================================== Tres Seaver tseaver(a)zope.com Zope Corporation "Zope Dealers" http://www.zope.com

2 1

Re: [Zope-dev] Zope 2.7.0-b2 startup error on windows
by Michael Long 11 Sep '03

11 Sep '03

> Try adding objects to your Zope, if that works and they stick around even after > you restart Zope, then this isn't so much of a problem. > > However, it's nasty, and I suspect you find you may not be able to store objects > to disk, so this does need fixing :-( > > Please file on http://collector.zope.org > I love the ease of adding zope instances in 2.7!! After some further investigation I have found that a product that I am developing is causing the error to appear. The product appears to work fine in zope. I can add it, create objects with it and the objects are persistent. Can someone give me some advice on how to go about debugging my product to identify the offending code? The error appears in the following environments: SuSE 8.2 -- Zope 2.6.1 WinXP Pro -- Zope 2.7.x Thanks, Mike

2 1

Re: [Zope-dev] Zope 2.7.0-b2 startup error on windows -- SOLVED
by Michael Long 10 Sep '03

10 Sep '03

I have found the offending code...he said sheepishly. In the __init__.py file I had the following: def initialize(context): ... import traceback; traceback.print_exc() instead of: def initialize(context): try: ... except: import traceback; traceback.print_exc() My thanks goes out to all who helped. Later, Mike > After some further investigation I have found that a product that I am > developing is causing the error to appear. The product appears to work > fine in zope. I can add it, create objects with it and the objects are > persistent. Can someone give me some advice on how to go about debugging > my product to identify the offending code? The error appears in the > following environments: > > SuSE 8.2 -- Zope 2.6.1 > WinXP Pro -- Zope 2.7.x > > Thanks, > Mike > > > _______________________________________________ > Zope-Dev maillist - Zope-Dev(a)zope.org > http://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope ) > >

1 0

AW: Caching prob with AHCM and headers
by Germer, Carsten 10 Sep '03

10 Sep '03

Well, sadly I didn't find the time to pursue the issue any further yet. We're kneedeep in work to get the system up and running and caching isn't top priority so far. I don't even know if someone patched the AHCM yet, simply haven't looked. But as someone suggested somewhere and you can see in http://mail.zope.org/pipermail/zope/2003-April/135101.html it works if you set the headers yourself. So if you need caching _now_ and AHCM is not fixed yet, change the object used so that it sets the headers correctly itself. I know that this solution is ikky but I don't know when I will have time to tackle the issue of a comfortable cache system. Though I know I will have to solve it sometime :) /Carsten > -----Ursprüngliche Nachricht----- > Von: Bjorn Stabell [mailto:bjorn@exoweb.net] > Gesendet: Mittwoch, 10. September 2003 13:07 > An: zope-dev(a)zope.org > Cc: jh(a)comunit.de; Carsten.germer(a)desy.de > Betreff: Re: Caching prob with AHCM and headers > > > Accelerated HTTP Caching Manager doesn't work out-of-the-box as this > thread reported, but the thread had no conclusion: > > http://mail.zope.org/pipermail/zope/2003-April/134800.html > http://mail.zope.org/pipermail/zope/2003-April/135059.html > http://mail.zope.org/pipermail/zope/2003-April/135101.html > > I'm running into the same problem now, also with Apache as the caching > reverse proxy, so I was wondering if there was any progress? Here are > some of my thoughts; > > It seems reasonable that without a Last-Modified or Etag header in the > response, that a client (and surrogate/ http accelerator) > will not have > any validators (e.g., If-Modified-Since or If-Match corresponding to > last modified and etag headers, respectively) in the request headers. > > It's probably right that dynamic code doesn't return a Last-Modified > header; instead, the RFC includes an Etag header, which is hash of the > content, which is much easier to use for dynamic code. The > Etag header > returned by Zope, however, looks very suspicous; it is empty. This > could perhaps be fooling the caching machinery? > > Perhaps the Etag header should just always be a hash of the content? > (Another system worth looking at: http://www.jpcache.com/) > > Bye, > -- > Bjorn >

1 0