May 2004 - Zope - Zope lists

RE: Leaking HTTP requests (was: RE: [Zope] Leaking Acquisition.Im plic itAcquirerWrapper)
by Jean-Francois.Doyon＠CCRS.NRCan.gc.ca 13 May '04

13 May '04

I do indeed use Localizer. I have a second instance of the Zope/Product configuration on a seperate machine, and so far I've been unable to replicate the problem. One thing I wanted to do was look at my REQUEST.set to see if that could cause problems. As for this global request patch, I'm not sure what you mean ... The test instance also uses Localizer and doesn't seem to suffer from the problem. The main thing here is that I'm not testing under load, just manually, trying to track down which actions might trigger refcounts. What's this patch exactly ? I do some monkey patching (Quite a bit actually, more than I should!) but nothing that messes with the request/response system. I have one place that uses REQUEST.set with wrapped objects ... based on what I've seen in some mailing list archives and from Dieter, this could very well be the cause ... (Not sure though, I used to have this on my previous site also, and never noticed it leaking ...) Thanks again for the assistance, it's proving very helpful :) J.F. -----Original Message----- From: Stefan H. Holek [mailto:stefan@epy.co.at] Sent: May 13, 2004 3:37 PM To: Jean-Francois.Doyon(a)CCRS.NRCan.gc.ca Cc: zope(a)zope.org Subject: Re: Leaking HTTP requests (was: RE: [Zope] Leaking Acquisition.Implic itAcquirerWrapper) This confirms my suspicion somewhat that the global request patch is to blame. IIRC Localizer uses this, doesn't it? Do you have the possibility to test without the (monkey?) patch, or without Localizer entirely? Huima has just recently reported pretty much the same bug and he is using PTS, which again utilizes the global request patch (I think). Stefan On Donnerstag, Mai 13, 2004, at 21:31 Europe/Vienna, Jean-Francois.Doyon(a)CCRS.NRCan.gc.ca wrote: > Oh, wait a minute ... > > Looks like I'm actually leaking HTTP requests! > > Class May 12, 2004 5:08 pm May 13, 2004 3:26 pm Delta > Acquisition.ImplicitAcquirerWrapper 7190 138920 +131730 > ZPublisher.HTTPRequest.HTTPRequest 352 18123 +17771 > ZPublisher.BaseRequest.RequestContainer 329 18095 +17766 > ZServer.HTTPResponse.ZServerHTTPResponse 338 18103 +17765 > > I saw some traffic on this same topic about archetypes & plone ... > > Hopefully I'll be able to track this down more easily than wrapper > problems. > > Still any input will be appreciated, tracking down memory leaks always > seems > like black magic to me :P > > Thanks, > J.F. > > -- The time has come to start talking about whether the emperor is as well dressed as we are supposed to think he is. /Pete McBreen/

1 0

RE: [Zope] Leaking Acquisition.ImplicitAcquirerWrapper
by Jean-Francois.Doyon＠CCRS.NRCan.gc.ca 13 May '04

13 May '04

Stefan, Hmmm actually yes ... Not quite as high as the wrapper, but higher than I'd expect, right this moment I've got 124219 for the wrapper and 48965 for DateTime ... I use standard user folders everywhere, I don't do anything special with them ... So I have one in the root and one in each of two CMF instances. I removed the "Members" folder ... I have about 4 or 5 users in each, most access to the site is anonymous. Although now that you say it, it'd be interesting to see if removing user-based/authenticated access solves the problem. I've been trying to replicate the problem on a second copy of the site without success ... But then I never authenticated against it other than through the ZMI ... I'm looking forward to your response now that you've peaked my curiosity :) Thanks for the help, J.F. -----Original Message----- From: zope-bounces(a)zope.org [mailto:zope-bounces@zope.org]On Behalf Of Stefan H. Holek Sent: May 13, 2004 1:00 PM To: Jean-Francois.Doyon(a)ccrs.nrcan.gc.ca Cc: zope(a)zope.org Subject: Re: [Zope] Leaking Acquisition.ImplicitAcquirerWrapper a) Are you also seeing high refcounts for DateTime objects? b) Which user folder do you have? Stefan On Donnerstag, Mai 13, 2004, at 17:02 Europe/Vienna, Jean-Francois.Doyon(a)CCRS.NRCan.gc.ca wrote: > I'm using Zope 2.7 w/ python 2.3.3 on RH7.3 with > - CMF 1.4.2 > - TranslationService > - Localizer > - mxmRelations > - ZPsycopgDA > -- The time has come to start talking about whether the emperor is as well dressed as we are supposed to think he is. /Pete McBreen/ _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

1 0

[Ann] AdvancedQuery 0.1 -- flexible catalog search with incremental multi-level lexicographic sorting
by Dieter Maurer 13 May '04

13 May '04

AdvancedQuery is a Zope product aimed to overcome several limitations of ZCatalog's native search function. Like ZCatalog's search, it supports elementary index searches. While ZCatalog can combine such elementary searches only by "and", AdvancedQuery allows them to be combined arbitrary with & (and), | (or) and ~ (not). While ZCatalog supports an efficient sorting via an index on one level, AdvancedQuery supports sorting via any level of (field) indexes. Moreover, it sorts the result incrementally -- only as far as you access your result. This can drastically speed up the time required for sorting. It uses Python's generators for this (and thus requires Python 2.2 or better). More information and download: <http://www.dieter.handshake.de/pyprojects/zope> -- Dieter

1 0

passing args from zpt to zpt
by Roy Mathew 13 May '04

13 May '04

Folks, Does anyone know if there is a way to pass arguments from a Page Template to another? The only way I can think of is to use SESSION to hold the values, and this seems clunky as I am then forced to remove those arguments from SESSION after. I need to do something like:  <span tal:define="global length python:23"> <metal:block tal:replace="structure here/nested.pt"/> </span>  <b tal:content="length|string:nothing was passed down"/> For whatever reason, even the global declaration doesn't seem to pass the value on down to nested.pt (my understanding of scoping is obviously flawed :). Thanks, Roy. __________________________________ Do you Yahoo!? Yahoo! Movies - Buy advance tickets for 'Shrek 2' http://movies.yahoo.com/showtimes/movie?mid=1808405861

3 2

dealing with file downloads and uploads
by Samuele Giovanni Tonon 13 May '04

13 May '04

hi, i have to work with files using zope, but i have some problem: i have one form in which you upload a file, then after the upload i have to parse (following instruction from the user) the file before inserting some of the data of it into a sql table. The problem comes when i have to pass the file from one form to another (this forms gives me instruction on how parse it) . I thought of some ways on how to deal with it but i don't know which one is better: - pass the file inside context/SESSION (but if the file is large and the connection is not good it could be slow) - create a temp file on to the filesystem (someone told me it coudl be dangerous and it's better not to use it) - create a temp file inside Zope (how? the module File seems not to have api that delete or truncate file isn't it ?) Also the same problem apply on to the download of a file that is like a csv (i found csvtool that handle this situation pretty well, however i'm curious on how to solve the problem with a any file) . Ah, i'd like to do everything without using dtml if possible. Any suggestion, code to look, web pages? Thanks Samuele

2 1

Product Generator
by Chris Bruce 13 May '04

13 May '04

Is there any product/code that allows one to generate python based products (Zope 2.7)? I am looking for something similiar to zclasses where I can input a list of the fields/type I want and it generates the python based product. I know this is wishful thinking, but looking for something before trying to re-invent the wheel. Thanks, Chris

2 1

Problem with permissions
by Clayton T. Dillard 13 May '04

13 May '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi All, I have just installed Zope version 2.7.0 and CMF version 1.4.3 on SuSE 9. I've followed the INSTALL directions for both Zope and CMF but when I try to start Zope I get the following error: Traceback (most recent call last): File "/opt/Zope-2.7/lib/python/Zope/Startup/run.py", line 49, in ? run() File "/opt/Zope-2.7/lib/python/Zope/Startup/run.py", line 19, in run start_zope(opts.configroot) File "/opt/Zope-2.7/lib/python/Zope/Startup/__init__.py", line 51, in start_zope starter.startZope() File "/opt/Zope-2.7/lib/python/Zope/Startup/__init__.py", line 230, in startZope Zope.startup() File "/opt/Zope-2.7/lib/python/Zope/__init__.py", line 46, in startup _startup() File "/opt/Zope-2.7/lib/python/Zope/App/startup.py", line 54, in startup DB = configuration.dbtab.getDatabase('/', is_root=1) File "/opt/Zope-2.7/lib/python/DBTab/DBTab.py", line 96, in getDatabase db = self._createDatabase(name, is_root) File "/opt/Zope-2.7/lib/python/DBTab/DBTab.py", line 113, in _createDatabase db = factory.open() File "/opt/Zope-2.7/lib/python/Zope/Startup/datatypes.py", line 172, in open DB = self.createDB() File "/opt/Zope-2.7/lib/python/Zope/Startup/datatypes.py", line 169, in createDB return ZODBDatabase.open(self) File "/opt/Zope-2.7/lib/python/ZODB/config.py", line 97, in open return ZODB.DB(section.storage.open(), File "/opt/Zope-2.7/lib/python/ZODB/config.py", line 128, in open quota=self.config.quota) File "/opt/Zope-2.7/lib/python/ZODB/FileStorage.py", line 232, in __init__ self._lock_file = LockFile(file_name + '.lock') File "/opt/Zope-2.7/lib/python/ZODB/lock_file.py", line 60, in __init__ self._fp = open(path, 'w+') IOError: [Errno 13] Permission denied: '/data/sszope/var/Data.fs.lock' Can anyone shed some light on why I'm having this error? Thanks in advance. Clayton T. Dillard SECURESPEED, LLC Office: (919) 557-5126 Mobile: (919) 395-9870 Fax: (919) 577-0943 Email: cdillard(a)securespeed.cc "Information Assurance and Security Solutions" -----BEGIN PGP SIGNATURE----- Version: PGP 8.0.3 iQA/AwUBQKLTGevKXu6K7abIEQL3vQCgtJEtKwFqNBGf3/e6htNq5+qw5KwAn0Gq qLiy9p/dRwnvfYadksIACAfd =zoCA -----END PGP SIGNATURE-----

2 1

Archetypes leak problem? Acquisition.ImplicitAcquirerWrapper references killing eventually Zope
by Heimo Laukkanen 13 May '04

13 May '04

First of all my apologies for long email, but this is an interesting problem to anyone. I appreciate any help and confirmations / views if anyone else can see the same. Setup: Python 2.3.3, Zope 2.7, Plone 2.0, Archetypes1.3b2 Problem that I am seeing now is on a service that yet does not have even a high load, but Zope keeps hogging steadily more and more memory, becoming slower and eventually coming to almost full stop - or in dire need of restart anyoway. Under small load this comes in few hours. What I have seen and debugged so far is that Acquisition.ImplicitAcquirerWrapper are running wild in Zope. Every time my Archetypes-object is accessed - atleast one new reference count is created. Even if that object is accessed again and again. By applying some force with ab or other tools numbers grow fast in just few thousand requests, below data from Zope Debug: Acquisition.ImplicitAcquirerWrapper 127 16986 +16859 Products.PageTemplates.Expressions.SubPathExpr 79 1503 +1424 ZPublisher.HTTPRequest.HTTPRequest 31 1249 +1218 ZServer.HTTPResponse.ZServerHTTPResponse 8 1226 +1218 ZPublisher.BaseRequest.RequestContainer 5 1223 +1218 And few thousand requests later. Acquisition.ImplicitAcquirerWrapper 127 36000 +35873 ZPublisher.HTTPRequest.HTTPRequest 31 4280 +4249 ZServer.HTTPResponse.ZServerHTTPResponse 8 4257 +4249 ZPublisher.BaseRequest.RequestContainer 5 4252 +4247 Products.PageTemplates.Expressions.SubPathExpr 79 2623 +2544 Products.PageTemplates.Expressions.PathExpr 45 1197 +1152 At this stage Zope processes have got approx. 20 percent of the full memory. Now to the setup that produces following behaviour. First of all there is a site: Plonesite SiteTree ( Archetypes object, folderish ) intranet ( Archetypes object, folderish ) And a PythonScript called sitetreetest, which accepts obj as parameter: --------------------------------------------- from Products.CMFCore.utils import getToolByName if obj is None: obj=context url_tool = getToolByName(context, 'portal_url') relative_ids = url_tool.getRelativeContentPath(obj) o = url_tool.getPortalObject() path_seq = [] for id in relative_ids: # this is much faster than restrictedTraverse and makes some sense, # if the user can't access the contents information, getattr() will fail # and hiding the breadcrumb because they dont have access? urk o = getattr(o, id) # o = o.restrictedTraverse(id) path_seq.append( o ) return path_seq ------------------------------------------------- Script will be called from a template: <table border="0" cellpadding="0" cellspacing="0" > <tr tal:repeat="itemi python: here.sitetreetest(here)"> <td tal:content="python: str(itemi)"> foo </tD> </tr> </table> And that is called in the context of intranet: /plonesite/SiteTree/intranet/template You can actually use event SimpleFolder or other folderish archetypes objects, since they provide similar results. It does not happen with Plone Folder. Code itself is similar to how Plone for example does breadcrumbs - so it should not itself be evil. What also makes this interesting is that reference count for Acquisition wrappers starts to run wild only IF the object does not have Access contents information permission for anonymous user set in workflow or otherwise. I've now debugged this for many, many hours and come to this point after excluding different other options away. Any help is appreciated as allways. -- -huima

2 1

dtml: how to check email address format?
by Jianping Zhu 12 May '04

12 May '04

The situation is that i want user to register. and i want to check the user-provid email address format. <dtml if "email=='' or....."> how can I check the email format? Thanks

3 2

product+catalog question
by Henny van der Linde 12 May '04

12 May '04

Hi, I have build a external product with its own classes for storing information. How can I get Zcatalog to index the data within these classes and how do I get Zcatalog to index certain parts of these classes (certain dictionaries). >From the ZMI, Zcatalog finds my product but it won't index for instance the tekst field wich is contained in the class. Classes are build on folder or objectmanager. regards, Henny

4 5