August 2001 - Zope-Dev

Memory Leak
by J�lio Dinis Silva 07 Aug '01

07 Aug '01

Hi, I'm running zope232src on Linux. My Data.fs packed has a size near 250 MB. My zope server takes 70Mb of RAM when started, ending after a week taking 1GB of RAM, and eventually restarts automatically (z2.py -Z). When it is at 1GB of RAM there is on top of refcounts in Control_Panel/DebugInfo the class BTrees.IIBTree.IIBTree with something like 3500 refcounts. Then there is OFS.Image.Image with 1000 refcounts and other classes with refcounts below this. I have a lot os catalogs in my Data.fs some with 10,000 others with 5,000 objects indexed. I've not converted all of my zcatalogs to the new Btree implementation with manage_convertBtrees. But now I'm thinking of doing that since I think the memory leak is in Btrees. I also have a BtreeFolder from the product BtreeFolder with 15000 folders created in it, but I dont think it could be from here. I'm doing debug to find this memory leak. Any hints if this memory leak could really be related with Btrees? Best Regards, JS _________________________________________________________________ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp

1 0

ZOracleDA
by Juan Javier Carrera Obrero 07 Aug '01

07 Aug '01

Hi, I am trying to install Zope in Windows 95 or Windows 98. It runs ok. However, I have installed the ZOracleDA product, but this product does not run correctly. This problem is caused because the library "buffer.dll" and "oci_.dll" are not enabled for Windows 9x. I have been compiled and trying to link the sources (in the "src" directory), but how the Makefiles and others files are prepared for Unix or Linux, I can not do it. Anybody knows to how to do it?, or anybody can send me these "dlls" for Windows 9x? Thank you very much.

2 1

In-memory persistence of non-serializable objects?
by Michael Strasser 06 Aug '01

06 Aug '01

I want to create a Zope object that can keep non-serializable objects (specifically sockets) in memory between requests. The idea is (in its simplest form): * On first request to the object a socket is opened and stored. * On subsequent requests the same socket is used. If one of these requests is to close the socket it is closed. If the socket has been closed by the other end an error is returned. I haven't found information about in-memory persistence of Zope objects and non-serializable objects can't be stored in ZODB. Have I missed something obvious? Thanks in advance. -- Michael Strasser Brisbane, Australia

3 2

Re: [Zope] can one pass values through the 'action' property?
by Florent Guillaume 06 Aug '01

06 Aug '01

[followups to zope-dev] Tommy, You're right, I did the tests and it is indeed a Zope problem. What happens is this. If you have: <form action="/action?row=3" method="POST"> <input type="submit" name="submit" value="submit"> </form> Then what gets sent by Mozilla to the web server is: POST /action?row=3 HTTP/1.1 Host: ... ... Content-Length: 13 submit=submit And indeed there is enough information here to build a correct REQUEST with both row=3 and submit=submit. (However note that with a GET method, Mozilla sends: GET /action?submit=submit HTTP/1.1 So the GET method wouldn't work here anyway.) The culprit is ZPublisher.cgi.FieldStorage, who only looks at the QUERY_STRING if it's a GET or a HEAD. If everybody else does it, maybe Zope should do it... A word about the spec: HTML-4.01 says: * If the method is "get" and the action is an HTTP URI, the user agent takes the value of action, appends a `?' to it, then appends the form data set, encoded using the "application/x-www-form-urlencoded" content type. The user agent then traverses the link to this URI. In this scenario, form data are restricted to ASCII codes. * If the method is "post" and the action is an HTTP URI, the user agent conducts an HTTP "post" transaction using the value of the action attribute and a message created according to the content type specified by the enctype attribute. So the "get" part isn't even clear, what happens if the action already contains a '?'... the spec doesn't say. Note BTW that an URI with two '?' wouldn't be legal. Cheers, Florent Guillaume Nuxeo > Florent, > > Ok, if it works in Coldfusion and ASP, then there isn't a problem with > 1. My browser > 2. HTTP spec > 3. Development language > > But if, now, I try to do the exact same thing in Zope, and it doesn't work, > then how could you NOT say that it isn't a Zope problem? I'm not totally > assuming that Zope is the problem, (assumptions can be deadly), but if it > has worked before, and doesn't work now with a new development language, > then where is the first place one should look? > > I'd be interested in input from developers who may have done this in PHP or > other languages. If it also works in them.... > > The only thing that I can think of is if Zope parses differently from other > languages. Meaning that if you choose to pass variables through the action > property, then you MUST use Get rather than Post. In the other languages it > didn't matter, but with Zope, it could potentially matter. I'll test it and > write back.

1 0

LDAP and local roles
by Florent Guillaume 06 Aug '01

06 Aug '01

I have a problem which can easily be solved by local roles: - several workgroups (each one is basically a folder) - for each workgroup, a list of users who are allowed to do stuff in it What I do is create a role WorkgroupMember, and in each workgroup assign a local role of WorkgroupMember to the users who are authorized. However I'd like all this information about workgroup membership to be stored in an LDAP directory, where it belongs. Currently I'm a bit stuck. What I envision would be something akin to LDAPLoginAdapter but whose effect would be on the local roles of a given folder. In my LDAP directory, I would have something like: dn: cn=WorkgroupMember, ou=workgroup1, ou=workgroups, dc=example, dc=com objectClass: groupOfUniqueNames uniqueMember: cn=bob, ou=people, dc=example, dc=com uniqueMember: cn=pete, ou=people, dc=example, dc=com uniqueMember: cn=joe, ou=people, dc=example, dc=com And the local roles in the folder would be parametrized by: groups base DN: ou=workgroups, dc=example, dc=com groups attribute: ou local role name attribute: cn login name attribute: cn user base DN: ou=people, dc=intercom, dc=gouv, dc=fr user RDN attribute: cn (all this can be factored somewhere) group: workgroup1 Well, that's how I see it. I have three questions: - is there a better way to do it ? - has someone done something like that ? Would a LDAPified Slave User Folder fit the bill ? (BTW, I should do a separate message about this, but Slave User Folder and LDAPLoginAdapter don't mix well, as LDAPLoginAdapter.getUsers returns only *cached* users... It could work if Slave User Folder only used getUserNames when listing available users) - If i do it with local roles, where and how would I plug the machinery to do this ? I need to replace the RoleManager, but I don't want to re-derive all classes. I could patch the folder to have a callable __ac_local_roles__, and I think things would mostly work (AccessControl.User does the right thing with callable __ac_local_roles__), but I'm not completely sure, beside that's ugly. Thanks, Florent Guillaume Nuxeo

1 0

RE: [Zope-dev] Weird ConflictErrors on startup
by Toby Dickenson 06 Aug '01

06 Aug '01

All of the standard products Chris W mentioned do this.... Zope keeps the product help index in a persistent BTree, which is updated automatically at product startup. > -----Original Message----- > From: Chris McDonough [mailto:chrism@zope.com] > Sent: 06 August 2001 17:36 > To: tdickenson(a)geminidataloggers.com > Cc: Chris Withers; zope-dev(a)zope.org > Subject: Re: [Zope-dev] Weird ConflictErrors on startup > > > It's generally bad form for a Product to try to modify a persistent > object at startup. Unless it's an _v_ attribute or something.

2 1

Copy, Cut and Cookies: a CopySupport.py question
by Jeff Sasmor 06 Aug '01

06 Aug '01

To someone with perhaps more ZZen than myself .... I am trying to understand CopySupport.py - It seems that objects that are cut or copied are stored in a cookie on the client. I could see how this would be easier than trying to associate some stored data item on the server with a particular client browser, but it also seems somewhat inefficient (esp. as regards upload and download times from the client) unless the object is relatively small in size and I also wonder if this scheme would break down completely for a big object, say something a few MB in size. Am I missing something ? Or maybe this is the only reasonably sane way to do it. TIA Jeff Sasmor jeff(a)sasmor.com www.netkook.com is an "open Zope CMF site"

2 1

Changes to Refresh
by Shane Hathaway 06 Aug '01

06 Aug '01

I've checked in a change to the way refresh works that should make it much more reliable. Instead of guessing at module dependencies and using the reload() function, it now just removes the appropriate modules from sys.modules temporarily. That way dependencies are resolved the natural way. For me, this corrected a lot of problems, at only a small cost. The change should be in the next Zope release, but if you're following CVS you can get the change right away. Please let me know if it fixes or breaks things. Shane

2 1

[Long] Advice solicited on Security, exUserFloder, and Components in Zope 2.4.x and forward?
by Dario Lopez-Kästen 06 Aug '01

06 Aug '01

Hello! A couple questions for ye Zope Deities, I am starting out in Python programming and Zope by adapting the exUserFolder product for usage with Zope 2.4.x and to our own security stuff. So far I have had limited luck, but I am slowly understanding how Zope UserFolders work and how exUserFolder works. If anyone has had any success in adapting exUserFolder for Zope 2.4.x please let me know, as you are probably better doing it than I am :-). I am adapting the exUserFolder for various reasons, the most important being that I need to manage amounts of 10k+ users with several different types of roles, that have to be calculated in some sintances. The setting here is a university building a Eduational Portal for it's students and staff. I would gladly accept any input of whether this is doable by adapting the exUserFolder or if there are any other means of doing it. I do not wnat to list 10k+ users in the management interface, nor do I want to have to list all possible, and current, roles each and every user might have. Eplanation of my rationale follows below. During this investigation I've looked at the ZopeDevGuide, and I am wondering if I can start using the new component interfaces with the current release of Zope. It is not clear that I cannot but it als is not clear if I can. Anyway, any help or comments are appreciated. --<begin longish problem explanation>-- Due to the nature of our infrastructure, I need to have a very clean separation of authentication, authorisation, user-mapping and user-information. I essence our needs mandate that we keep a map-database between various keys identifying users in various systems, where Zope works as the middleware. Our "authentication service" are actually three different auth services (try one, if fail try next, etc) that we need to employ during a trasition phase (we'll end up using DCE in the end for all centralised authentication services). The mapping service helps us map the username to various other keys needed to iteract with some of our legacy systems. We maintain a database mapping username with, for example, social security number. This way, when you log in, you only use your username and passwd and then the system automagically knows your social sec nr and a) gives you personalised info from the university and b) prevents you from accessing other user's personal info. The auth service will be used to allow or restrict access to various parts of the system. For instance, it is desirable to restrict certain features in a course homepage system to people that are actually registered as participants in the course. For this we want to use a system of "calculated roles" in combination with what I, for lack of a better name, would call "adapted queries" (as in "return only stuff for user x"). So I am thinking of finding a way of providing Zope with a list of acceptable roles, say, as the result of a ZSQLMethod. The user-info service would provide us with other user-properties such as Full Name, Home Adress, etc. The problems I am having so far is that I cannot even get etcAuthSorce working. I am now trying with usAuthSource and have had some luck with it, but nothing that works quite right just yet. --<end longish problem explanation>-- Any helps is, as always, appreciated. Sincerely, /dario cc-ing zope-edu-user, eurozope - -------------------------------------------------------------------- Dario Lopez-Kästen Systems Developer Chalmers Univ. of Technology dario(a)ita.chalmers.se ICQ will yield no hits IT Systems & Services

1 0

Re: [Zope-dev] Help on Zope security needed ...
by Oliver Bleutgen 06 Aug '01

06 Aug '01

Hi Joachim, > No, it shouldn't. > Am starting to think that the Zope security model implementation is a bit > "strange". So, actually I thought that _my_ brain is going wrong here, but if it isn't, this is IMO a very serious bug. I just couldn't believe that something serious like that could have gone through up to zope 2.3.3 without anybody noticing, when it is indeed not wanted behaviour. >What I'd need in practice is a security model that > a) is completely predictable (that's what > http://dev.zope.org/Wikis/DevSite/Proposals/SecurityJihad is working on) > b) would normally make sure that somebody in a subfolder can not get > access > to anything that is explicitly protected in the parent folder I would think not follwing a) and esp. b), the software has a bug, or the documentation. I for one can't find anything in http://www.zope.org/Members/michel/ZB/Security.dtml which would warn about the above described behaviour. > c) would on the other hand offer the possibility to bind access to a role, > regardless where in the folder hierarchy somebody gets the role. No, I wouldn't like this, I always thought that containment before context should also apply to security. > The problem is with acquisition: If I have an "editButtonsBar" widget in > my > root directory, I'd like to make it available to ALL Editors, not just to > the ones who have editor roles in root. Do you want to make it available or editable? > On the other hand, if I have a standard header or footer that should not > be > overriden in a subfolder, this should be possible. This means I'd not only > need a permission that can not be bound to roles in the subfolder, but > also > a mechanism to prevent overriding the object ... Isn't this a typicial problem for a zclass or product or whatever? Even if I couldn't overide standard_html_header, i.e. create my own methods with that name, I could always change the places where it is called to call my_own_header instead. cheers, oliver

4 3