- Zope-Dev - Zope lists

ZPatterns Image Archive Project
by Michael R. Bernstein 31 Mar '01

31 Mar '01

Hello all, Background ---------- I am building an Image Archive Zope application using ZPatterns. I originally was trying to convert the Photo Product to ZPatterns in such a way as to still have it work as a normal product, but that turned out to be too complex. So, I am building the application fresh. So far, I have a RackImage Python product that subclasses from Image and DataSkin which seems to work fine. I can create, upload, view and delete the RackImages. Now I am working on a ArchiveImage ZClass that holds 'meta' information about an image, such as the description, a date, and keywords. I want to have one Rack for each image size that I want to store. I realize that I need to tie the Racks together in the specialist using SkinScripts, but I had a question that is a bit more fundamental: Where do I put the separate Racks? do I create a specialist for each Rack (seems unneccessary) or do I put several Racks in the same Specialist? If the latter, I'm unsure how to use DTML or PythonScripts to specify which Rack to create a new item in. The DTML I am using to instantiate RackImages in the Rack for example, is as follows: <dtml-let ni="newItem(REQUEST.id)"> <dtml-call "ni.manage_edit(REQUEST.title,'')"> <dtml-call "ni.manage_upload(REQUEST.file)"> </dtml-let> But this code only works for a Rack named defaultRack, which I can obviously only have one of. Any advice on this would be greatly appreciated. Thanks, Michael Bernstein.

2 1

access db linux
by danchik 31 Mar '01

31 Mar '01

does anyone know if there a way to have an access db to be placed onto linux box with zope running on it and have it be able to access that db????... I don't really want to convert the entire database.....

4 3

ANNOUNCE: Zope 2.3.1 final released
by Brian Lloyd 30 Mar '01

30 Mar '01

Hello all, Zope 2.3.1 final has been released - you can download it from Zope.org: http://www.zope.org/Products/Zope/2.3.1/ Zope 2.3.1 contains fixes for a number of issues that have come up since 2.3.0, including the fixes for the issues addressed by the 02/23 and 03/08 hotfixes. It also contains a number of changes to address issues with ZCatalog. New data structures have been introduced to reduce the number of conflict errors caused by object indexing under some types of usage, and to reduce the amount of object database bloat that occurs when indexed objects change frequently. The changes to ZCatalog have been designed so that newly created ZCatalog objects will take advantage of the new data structures. Existing ZCatalog objects will continue to work with the old data structures, but will not gain any of the benefits of the new code. It is possible to upgrade an existing ZCatalog object to use the new data structures by following the How-To at: http://www.zope.org/Members/mcdonc/HowTos/UpgradeToNewCatalog Many thanks go to those who helped exercise the new code and the catalog upgrade process during the beta cycle. Many thanks as well to Jim Fulton, Chris McDonough and Andreas Jung for their hard work and the prodigious amount of unit tests produced for the new (and old) Catalog code! With the 2.3.1 release we have added a Linux Itanium binary distribution. Note that the Itanium binary is bundled with Python 2.0 instead of Python 1.5.2. This release (Zope-2.3.1-linux2-ia64-alpha.tgz) is named with "alpha" because it is the first for the platform. All unit tests pass and moderate testing has found no problems (actually, any problems you find are more likely to be issues with Zope on Python 2.0 rather than issues with the platform). **IMPORTANT NOTE** for users of the source release or users of the "diff" update releases Users of the Zope source releases should note that this release contains changes to C code. If you are using a source release (or the source diff update), be sure to run 'w_pcgi.py' or 'wo_pcgi.py' as appropriate before starting Zope to ensure that your binaries are rebuilt. Also, there has been some recent work on getting Zope to run on the Mac OS X platform. As a result of this work, the names of certain C extension modules were changed to avoid symbol name clashes on systems with a global dynamic module namespace. The result of this is that if you install a diff update over an existing installation, you need to do a few things manually: - If you are using a source release and apply the source update, visit the directories 'lib/python/BTrees' and 'lib/python/Shared/DC/xml/pyexpat'. In each, run 'make clean' and remove any .so files in those directories before rebuilding the extensions. - If you are running a binary release and have applied the diff update for your platform, before restarting Zope visit the 'lib/python/BTrees' directory and remove any .so (or .pyd) files that do *not* have names beginning with an underscore and in the directory 'lib/python/Shared/DC/xml/pyexpat', remove the file 'pyexpat.so' (or .pyd) For more information on what is new in this release, see the CHANGES.txt and HISTORY.txt files for the release: http://www.zope.org/Products/Zope/2.3.1/CHANGES.txt http://www.zope.org/Products/Zope/2.3.1/HISTORY.txt Note that we have also posted "diff" updates as .tgz files that will let you easily upgrade an existing 2.3.x site. These updates are available for those currently using the 2.3.x source release or the 2.3.x binary releases. To apply a differential update to your site: - download the appropriate .tgz file from zope.org - shutdown your Zope process - copy the .tgz to your Zope directory and extract it - SEE THE NOTE ABOVE!! - run w_pcgi or wo_pcgi *if you are not using a binary release* - restart your process Brian Lloyd brian(a)digicool.com Software Engineer 540.371.6909 Digital Creations http://www.digicool.com

1 0

Problem with Hotfix 2000-10-11 on 2.1.6
by Gregor Hoffleit 30 Mar '01

30 Mar '01

I'm having trouble installing Hotfix_2000-10-11 on Zope 2.1.6, although the README says: "The hotfix will work for all versions of Zope 2.2.0 and higher." If installed on 2.1.6, the product shows up as broken, since it tries to run "from OFS.ObjectManager import aq_base", which fails, since aq_base wasn't available in ObjectManager before 2.2.1 (!). Therefore I guess the Hotfix won't work for any versions prior to 2.2.1. According to the README, those versions are still vulnerable. Could somebody give me a hint if and how it's possible to backport the Hotfix to Zope 2.1.6 ? Gregor

3 4

RE: [Zope-dev] ZCatalog
by Marco Nova 30 Mar '01

30 Mar '01

> > I'm a newbie about python and zope (about a week and a half > till now), so > > probably most of what I say now is probably wrong. > > > > I was trying to customize the ZDiscussion product with file upload, > editing > > of previous posts and deletes of old posts and I guess I've > find a bug in > > the ZCatalog.uncatalogObject method. > > Thanks for the report! > > This is already fixed in CVS. you're welcome. > > Or if we want to handle situations where the unindexs are > unsuccessfull, > we > > have to rewrite better the method with the use of transactions. > > We needn't worry about manually managing transactions here > because the ZODB > takes care of it for us. So if a a method raise an exception all the things done (that involve zodb) in the method will be rolled back ? - mn

2 1

Re: [Zope-dev] (ZPatterns) Speeding up Skinscripts
by John Eikenberry 30 Mar '01

30 Mar '01

Phillip J. Eby wrote: > Congratulations, you're the first person (that I know of, anyway) to hit a > wall here. However, I think you may be barking up the wrong tree on your > profiling. See below. What a dubious honor. > SkinScript expressions execute using Zope security checking. If you're > using a "WITH QUERY SQLmethod() COMPUTE attributelist" statement, security > checks are applied to both the query expression and to every name > referenced in 'attributelist'. So the longer the attribute list, the > slower the execution. Although, now that I think of it, eval() is not used > if you use a pure attribute name list like "name1,name2,name3", so the > problem probably isn't on the attribute list side. This makes it much less > likely that the security machinery is to blame (unless you have a very slow > user folder which also doesn't cache well, and you haven't given your > SkinScript proxy roles to shortcut the security checks.) Yeah... I just added proxy roles to all the skinscripts and it didn't make that much of a difference (maybe a small one though). > Now, for your SQL query... is it by any chance dynamically generated in a > complex way? I mean, more complex than an sqlvar or two? Keep in mind > that when Zope caches SQL queries it does it by generating the text of the > query and using it as a cache key. That means any DTML in your query will > be executed each time. It'd have to be pretty complex, or else there'd > need to be lots of slow security lookups, to get the kind of poor > performance you're seeing, though. In the sql method in question has 1 dtml-sqlvar in it. Everything else is pre-rendered (there's a method that generates the sql methods). > 1. If you're not using LoginManager as your user folder, and maybe even if > you are, your SkinScripts should have proxy roles. Proxy roles can slash > the overhead of performing security checks, and are usually appropriate for > SkinScripts because SkinScript is "internal" to the object and Zope > security will apply to the results of the SkinScripts as well. Most of my tests have been with the builtin user folder. But as I mentioned above, proxy roles don't help much. > 2. Why are you retrieving 200 dataskins? Is this all in one transaction? > If so, you may have a design issue. Multi-object operations should > generally be implemented as domain-specific methods on the Specialist, so > they can be implementation-specific (and thereby take advantage of speedups > like a ZCatalog index or specialized SQL queries). In other words, you > probably want to iterate directly over a 200-row query result from an > SQLMethod, rather than retrieiving 200 DataSkins. (You're accessing over > 10 times as many DataSkins in one transaction as our most complex ZPatterns > application touches in its most complex type of transaction, which involves > about 4 or 5 Specialists at once!) Pretty much, if you're displaying a > list or report or doing some kind of summary analysis or mass update, it > belongs in an implementation-specific method in the Specialist. Yeah... this is mainly a design flaw. I and others at my company were working on various parts of this project in parallel. I was working on the base level ZPatterns model. They were working on client apps. We missed the communication bus at some point and they got to thinking of Specialists as DB tables. Something that needs to be addressed in the future, but it doesn't help me now. :( > 3. If you *really* need 200 dataskins in one transaction, and the overhead > is *really* in SkinScript, then you can bypass the overhead by writing an > AttributeProvider of your own in Python. I'm guessing, however, that > SkinScript per se is not the source of the slowdown, and you need to look > more closely at the things the SkinScript is calling. I think I'll have the developer just use a SQL method to get the data. This is how it should be done anyways... Thanks for the reply. -- John Eikenberry [jae(a)kavi.com] ______________________________________________________________ "A society that will trade a little liberty for a little order will deserve neither and lose both." --B. Franklin

1 0

(ZPatterns) Speeding up Skinscripts
by John Eikenberry 30 Mar '01

30 Mar '01

We have a fairly large and complex app framework built on ZPatterns. It uses MySQL for storage and the standard Specialist/Rack/DataSkin setup with skinscripts for attributes and triggers. We've found that the speed of getItem is a bit slower than we need. For instance retrieving 200 dataskins takes about 8 seconds on a P2-300. After profiling and digging around I think I've found the primary bottleneck. Its the running of eval() on the skinscript's python expression (stored in the Compute class as _fromex and Triggers as callexpr). Note that this becomes the bottleneck after the SQL Method gets cached. The query to the DB takes the most time on the first hit, but after its been cached it takes very little time. The optimization I've been looking at is changing the code from storing a string and eval()ing the string to using compile() at save time and exec() when evaluated. Profiling these 2 ways in little test programs seems to indicate about a 2.5x speedup. Not huge, but combined with better hardware should be enough. But I'm curious why this avenue wasn't taken to begin with. Seems like the way to do it to me. Am I missing something? -- John Eikenberry [jae(a)kavi.com] ______________________________________________________________ "A society that will trade a little liberty for a little order will deserve neither and lose both." --B. Franklin

4 4

Re: [ZWeb] 'All Hotfixes' page (was Re: [Zope-dev] Problem with Hotfix 2000-10-11 on 2.1.6)
by Martijn Pieters 29 Mar '01

29 Mar '01

On Thu, Mar 29, 2001 at 01:27:08PM -0500, Shane Hathaway wrote: > > Have a look at http://www.zope.org/Products/Zope/hotfixes, I think it is > > exactly what you need. > > Huh? You repeated the URL he supplied... and I'd like to know if there > is indeed a better URL. Duh. Copy, paste, send. Who cares about editing the URL..... :0 The correct URL is: http://www.zope.org/Products/Zope -- Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------

2 1

Re: [Zope-dev] ZCatalog phrase indexing revisited
by Dieter Maurer 29 Mar '01

29 Mar '01

Oren Yosifon writes: > ... phrase searches ... > Can anyone help me ? URL:http://www.dieter.handshake.de/pyprojects/zope/near.pat is a patch that made near searches work ages ago for me. The patch is old (against Zope 2.1.6). As todays ZCatalog is quite different, it will not be easy to apply the patch to modern code. You only have a chance, when you are quite familiar with Python. Dieter

1 0

Re: [Zope-dev] Zope/OpenBSD: Too many open files
by Dieter Maurer 29 Mar '01

29 Mar '01

Ricardo Bermell writes: > Zope (started from a shell) in OpenBSD gets locked > too easyly because of the low default limit for > open file descriptors (64). Most operating systems allow you to increase this number (either at runtime or through a kernel recompilation). Dieter

3 2