October 2003 - Zope-Dev

New Zope Database Adapter Test Suite available
by Chris Withers 31 Oct '03

31 Oct '03

Hi All, I've just finished writing a test suite for Zope DA's in general, based on what I saw in Shared.DC.ZRDB.Connection and Shared.DC.ZRDB.DA. This is available from: http://cvs.zope.org/*checkout*/Products/DCOracle2/test/test_zopeda.py An example of how to use it (for ZOracleDA in this case) is available from: http://cvs.zope.org/*checkout*/Products/DCOracle2/test/test_zoracleda_basic… There are instructions for running it at the bottom of: http://cvs.zope.org/*checkout*/Products/DCOracle2/README I'd be very interested if users of the other database adapters could see how this does against their products. If anyone wants help to do this, please mail me off list. If this proves useful, I'd like to check it into the core Zope CVS, what do other people think? I do have some questions, though. DA's return a Shared.DC.ZRDB.TM subclass when they are called, this has lots of transaction-ish methods. I currently check that these are there, but does anyone know how I could check that they do what they're supposed to? Also, if anyone can see any methods on any objects that I'm not currently testing, but that should be tested, please let me know! cheers, Chris

1 0

Re: unicode and structured text (was Re: unicode oddities )
by Andrey Lebedev 31 Oct '03

31 Oct '03

On Fri, Oct 31, 2003 at 04:40:53PM +0100, Godefroid Chapelle wrote: > At 16:20 31/10/2003, H. Turgut Uyar wrote: > >>>There is a patch for structured text that fixes this behaviour. You can > >>>find it at: http://zope.org/Members/kedder/stx_18n_patch > >>I just downloaded it, applied it to my french setup and it works :-) > >>Thanks a lot. > >>Did you send it to the zope-dev list, it could make a lot of sense in > >>2.7... if it is still time. > > > >I agree. But it's not my patch, I thought the original author should do > >the contacting. Actually, it was announced on the zope.org site several > >months ago so I thought the zope developers would have heard about it. > > > >-- Turgut > > Hi Andrey, > > how secure do you feel about proposing to integrate your structured text > i18n patch in 2.7 ? With pleasure, this patch was installed on our copany's production server for half a year and worked fine. Lots of people had reported the same. However i feel it needs a little bit more testing, especially with zwiki and plone. I had one report about problems with plone, but i didn't managed to reproduce it on my system with the latest versions... The problem was an exception "ASCII decoding error: ordinal not in range(128)", when he tried to edit some properties in plone document. Maybe it was an already fixed plone bug, but who knows... If you know a good way to attract testing forces to the patch, please suggest:) Another thing a want to do before merging with zope is to consult with zope gurus out the right way to do the trick: Current (0.9.9) version of patch makes StructuredText to return unicode object, the earlier one (0.9.0) used to return plain utf-8 encoded string. I thought (and think) it is better to return a unicode object, since zope publisher could convert it then to any other encoding. However the guy with problem mentioned above reported, that the problem disappeared when he switched to 0.9.0 patch. So, if someone has any thoughts about it, please tell (i hope i've described clearly :) ). and BTW, what is the best way to propose a patch? > > BTW a lot of thanks for your solution to a problem that was biting my users. I'm always glad to hear it. Thanks :) -- Andrey Lebedev aka -.- . -.. -.. . .-. Software engineer UAB Mikromarketingas (http://micro.lt) Homepage: http://micro.lt/~andrey/ Jabber ID: kedder(a)jabber.ru

1 0

ThreadLock module?
by Chris Withers 31 Oct '03

31 Oct '03

Hi all, Anyone know what the difference is between Zope's ThreadLock module and the standard python Lock objects from the threading module? Chris

2 1

Re: [Zope-CMF] Better MemberData search for CMF
by Florent Guillaume 31 Oct '03

31 Oct '03

[The main discussion is focused on CMF, please keep zope-cmf(a)zope.org in the Cc.] Basically we need a search API. I'm Cc-ing zope-dev, exuserfolder-devel and plone-developers (for GRUF) because we're talking about user folders. The last part is especially of interest to you guys. Jens wrote: > Florent Wrote: > > Currently CMF's MemberDataTool has > > searchMemberData(search_param, search_term, attributes=()) > > but it can only do a search on one attribute. It returns a list of > > dictionaries. > > > > CMFLDAP's MemberDataTool inherits the one from CMF, and so actually > > does the search only in cached members. This is a big problem for me. > > The way it is done in CMFLDAP stems from how the LDAPUserFolder reacts > to calls such as getUsers, which will retrieve all users. Calls like > that are inappropriate and even dangerous for any user folder that has > a storage with large numbers of records. Zope (and possibly the storage > backend used by the user folder) will come to a standstill if all of a > sudden thousands of records come flooding back and they all need real > Zope user objects created for them. > > The LDAPUserFolder has a notion of caching users to avoid the speed > penalty caused by having to go to LDAP. I made the conscious decision > to implement getUsers in a way that will only look at the > currently-cached users. Anything else is potentially disruptive for the > running Zope or the LDAP backend. Yes I understand the reasons for this behavior. Still it's painful, and is a proof we need a search API. > > Plone (in MembershipTool) has > > searchForMembers(**kw) > > that could search on arbitrary attributes but actually only does it on > > name, email, roles, and last_login_time. It returns a list of user > > objects. > > > > CMFLDAP's PloneLDAPMembershipTool seems to be more intelligent than > > its MemberDataTool and do the full LDAP search, but I'm not sure. > > The PloneLDAPMembershipTool's searchForMembers is different in response > to specific Plone needs. In my opinion it still sucks really bad. If > you look at the code in searchForMembers in the standard Plone > membership tool you will see that it does the "dangerous" thing of just > grabbing all users from the user folder and then looking at each user > to see if the search criteria fit. > > There's another item that makes searching members in a CMF-based portal > more complicated: Users that you find in the user folder are not > necessarily members, even if the user seems to have the Member role. > You always end up having to introspect the MemberData tool to see if it > has a member data wrapper with the same ID. Yeah maybe we also need a MembershipTool.hasMember() method ? It's better than doing 'id in mtool.listMemberIds()' ... > > I'd like to propose a standardized API on which we can agree for the > > future: > > MemberDataTool: > > searchForMembers(mapping, maybe_some_options=None, **kw) > > It would search on all the keys of mapping+kw. > > The options would be there to ask for a substring or exact match. To > > be decided later. > > I'd like it to return a list of users, but this may be problematic > > from a security point of view. What do you think? Otherwise we could > > make it take an attributes=('email', 'fullname') args, and return a > > dict with at least the specified attributes. > > If you want to go the "sequence of dictionaries" route then the > properties as defined on the MemberData Tool should be looked up to > define a set of standard attributes to return, meaning attributes above > and beyond the user name. Yes. > > A side problem is that this method will have to have knowledge of how > > to do a search in the user folder. And there are lots of those, with > > nowhere an standardized API in sight to do search either :-( (actually > > I don't know any that can do a search appart from LDAPUserFolder). > > Maybe we could take the oportunity to define one ? Searching for > > userids, roles, groups (for me), is a prerequisite for me. > > A standard API for such searches on the user folders would be a > prerequisite for any of this so that the user folder, which is the only > link in this chain that really knows how to make the search efficient, > can do so. That would completely eliminate that nasty pattern of "I > will grab all users myself and then look at each user object". I agree, so I'd like to propose an API for this too. I haven't thought about it much. LDAPUserFolder has findUser(search_param, search_term, attrs=[]) but it only searches one one term. So I propose: searchUsers(mapping, attributes=(), other_options=..., **kw) it would work in the same manner as searchForMembers above. As above, do we make it return a sequence of user objects or a sequence of dictionaries ? In any case we'll need a way to only return user ids. Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com

2 1

Zope 2.6+ pyton 2.3 ? and "Bad Call" on zope 2.6.2/python 2.1.3 with profiling turned on.
by Romain Slootmaekers 31 Oct '03

31 Oct '03

Yo, I have an elaborate piece of python code that works fine, but with profiling turned on, it barfs a "Bad Call" message. When google-ing for a solution, the only thing I could find was a http://mail.python.org/pipermail/python-bugs-list/2003-June/018510.html There's a follow up stating it's "already fixed", and does not occur in Python 2.2.x and beyond. So now my questions are: - how difficult is it to get a zope 2.6.2 (or 2.7) working with a python 2.2.x or 2.3. on a windows platform? (I read up on the changes between 2.6 and 2.7 and it does not seem to change python version.) - I believe there's a concensus that the next major 2.X release should jump immediately 2.3, but is there a timeline ? - or is there a workaround/solution to the "Bad Call" problem? This all to see if I should invest time in getting a 2.6 or 2.7 +python 2.3 up and running or I should do my own time measurements/profiling effort. TIA, Romain Slootmaekers.

4 5

Zope and interbase
by Paulo Lobo 30 Oct '03

30 Oct '03

Pessoal, é o seguinte: Guys, i have a problem... i'm working on a site with Zope and Interbase, and sometimes some strange things start to happen... when a lot of requests are made, for example, if someone press F5 and keep it pressed, zope locks and interbase too if we acess a page that doesn't uses Interbase, in the same server, zope doesn't locks, even if we try to acess it with more than one user at a time i think that's something wrong with my Interbase connector. I'm using ZFirebird, because we already had this problem with L Interbase Connector. If someone could show me a better Interbase connector ou another form of solving this problem, i would be glad... thanx in advance

2 1

Catalog performance
by zhimin＠iss.nus.edu.sg 30 Oct '03

30 Oct '03

John Barratt wrote: > If you can't use catalog metadata as Seb suggests (eg. you are actually > accessing many attributes, large values, etc.) and if indeeed memory is > the problem (which seems likely) then you can ghostify the objects that > were ghosts to begin with, and it will save memory (unless all those > objects are already in cache). > > The problem with this strategy though is that doc.getObject() method > used in your code activates the object and hence you won't know if it > was a ghost already or not. To get around this you can shortcut this > method and do something like : > > docs = container.portal_catalog(meta_type='Document', ...) > for doc in docs: > obj = doc.aq_parent.unrestrictedTraverse(doc.getPath()) > was_ghost = obj._p_changed is None > value = obj.attr > if was_ghost:obj._p_deactivate() Just my 2 cents observation... I ran this code and monitored the page "Cache extreme detail" in ZMI > ControlPanel > DebugInfo. With this method, the object was not loaded. However the intermediate objects that the unrestrictedTraverse() passed by were loaded into memory. e.g. If doc.getPath() is '/x/y/z/myobject', myobject was not loaded but x, y, and z were loaded into memory. I also tried the method suggested by Seb. This did not load myobject as well as x, y, and z into memory: http://mail.zope.org/pipermail/zope-dev/2003-September/020450.html The information on deactivating object into ghost state is very helpful. Thanks! cheers, zhi min

10 37

Re: [Zope-dev] Windoz file naming
by alan milligan 30 Oct '03

30 Oct '03

OK - thank you both for your help. It's time for: > perl -pi e 's/Currency/ZCurrency;g' `find . -name "*.py"` ;) >From: Anthony Baxter <anthony(a)interlink.com.au> >Reply-To: Anthony Baxter <anthony(a)interlink.com.au> >To: "alan milligan" <alan_milligan(a)hotmail.com> >CC: zope-dev(a)zope.org >Subject: Re: [Zope-dev] Windoz file naming Date: Thu, 30 Oct 2003 17:24:39 >+1100 >MIME-Version: 1.0 >Received: from crown.off.ekorp.com ([203.63.137.225]) by >mc4-f31.hotmail.com with Microsoft SMTPSVC(5.0.2195.5600); Wed, 29 Oct 2003 >22:27:28 -0800 >Received: from crown.off.ekorp.com (localhost [127.0.0.1])by >crown.off.ekorp.com (8.12.10/8.12.10) with ESMTP id h9U6RPwD026096;Thu, 30 >Oct 2003 06:27:25 GMT >Received: (from uucp@localhost)by crown.off.ekorp.com >(8.12.10/8.12.10/Submit) id h9U6RPP5026055;Thu, 30 Oct 2003 06:27:25 GMT >Received: from bonanza.off.ekorp.com(192.168.41.16), claiming to be >"localhost.localdomain" via SMTP by mx1.ekorp.com, id smtpdAAAiPaO3Y; Thu >Oct 30 06:27:18 2003 >Received: from localhost.localdomain (localhost.localdomain [127.0.0.1])by >localhost.localdomain (8.12.10/8.12.8) with ESMTP id h9U6Od9Q031000;Thu, 30 >Oct 2003 17:24:39 +1100 >Received: from arbhome.com.au (anthony@localhost)by localhost.localdomain >(8.12.10/8.12.8/Submit) with ESMTP id h9U6OdVd030996;Thu, 30 Oct 2003 >17:24:39 +1100 >X-Message-Info: JGTYoYF78jFFkSPI26F/WteBIBm95Syi >Message-Id: <200310300624.h9U6OdVd030996(a)localhost.localdomain> >X-Authentication-Warning: localhost.localdomain: anthony owned process >doing -bs >X-Mailer: exmh version 2.5 01/15/2001 with nmh-1.0.4 >Organization: dis- X-Complaints-To: my cat In-Reply-To: ><BAY8-F73lbAqvxI6Wzm0002bb37(a)hotmail.com> Return-Path: >anthony(a)interlink.com.au >X-OriginalArrivalTime: 30 Oct 2003 06:27:29.0044 (UTC) >FILETIME=[E46E5540:01C39EAE] > > > >>> "alan milligan" wrote > > Guys, > > > > I've just run into a problem and wondered if there was an elegant > > resolution. > > > > My BastionBanking package contains: > > > > currency.py - pure python > > Currency.py - Zope derivation > >The "typical" approach in Python is to have, e.g. Tkinter, and _tkinter. > >But if it's a zope derivation of currency, why not call it zcurrency? > _________________________________________________________________ Find your perfect match @ http://personals.xtramsn.co.nz with XtraMSN Personals!

1 0

Windoz file naming
by alan milligan 30 Oct '03

30 Oct '03

Guys, I've just run into a problem and wondered if there was an elegant resolution. My BastionBanking package contains: currency.py - pure python Currency.py - Zope derivation The idea is that using the setup.py script, you can load currency.py into your Python modules and use it outside Zope if you wish. However, under Windoz, it doesn't recognise the different file names and wont allow them to both reside in the same directory! I had intended to use this technique widely to have a single distribution stream for both pure Python and Zope-wrapped modules. TIA Alan _________________________________________________________________ Surf the net and talk on the phone with Xtra Jetstream @ http://www.xtra.co.nz/products/0,,5803,00.html !

3 2

Controlling start en finish of a request
by Dario Lopez-Kästen 29 Oct '03

29 Oct '03

Hello, I need help with ideas on how to solve my little problem here. I need to sort of be "more in control" of a request, meaning that at the beggining of a request I want to set some variables and call some python scripts and at the end of a request I want to set soem other variables and call some other python scripts How would I be able to accomplish that? We sort of have semi-solved it by setting and calling from standard_html_header and standard_html_footer, but that is not enough. IS there a way of achieving what I want to do? I was told on IRC to do look into TransactionManager but I don't know how to make this availabel Zope-wide. Any ideas are appreciated. Thanks, /dario - -------------------------------------------------------------------- Dario Lopez-Kästen, IT Systems & Services Chalmers University of Tech.

2 1