- Zope - Zope lists

Searching multiple catalogs
by Duncan Booth 18 Mar '05

18 Mar '05

I wrote some code to search multiple catalogs and merge the results, and I've got it working, but I don't really understand why LazyCat works the way it does, so I'm concerned in case there are situations where what I've done won't work properly. Normally when you search a catalog you get back a LazyMap which applies a function to the appropriate result when you call __getitem__ so that it appears to contain a list of brains but actually only creates them as needed. If you call searchResults with _merge=0 then you can call mergeResults with multiple result sets and get back a LazyCat. Calling __getitem__ on the LazyCat returns a 3-tuple and you can construct a brains object from the tuple. What I don't understand is why LazyCat doesn't do that final step of constructing the brains itself? In particular, does it mean that there is some situation that I haven't hit yet where something else gets returned and the brains cannot be constructed this way? Since I want to use the same template to display search results whether they come from one or several catalogs, I wrapped my result in a class to convert the LazyCat elements into brains, and as I said above this seems to work but I'm worried that since I don't really understand the thinking behind this code there may be edge cases which break. Can anyone help clarify this? The code I've ended up with: class LazyCatToBrains: '''Lazycat returns tuples instead of converting to brains so we need to convert the tuples into brains ourselves when a LazyCat item is accessed. ''' __allow_access_to_unprotected_subobjects__=1 def __init__(self, lc): self.seq = lc def __getitem__(self, index): v = self.seq[index] if isinstance(v, tuple) and len(v)==3: v = v[2](v[1]) return v def __getattr__(self, name): return getattr(self.seq, name) ... the further down in the main class ... # Based on code by Casey Duncan security.declarePublic('queryMultipleCatalogs') def queryMultipleCatalogs(self, request, *zcatalogs): results = [] for zcat in zcatalogs: results.append(zcat._catalog.searchResults(request, _merge=0)) sorted = request.has_key('sort-on') or request.has_key('sort_on') reverse = ((request.get('sort-order','') or request.get('sort_order','')).lower() in ('reverse','descending')) results= mergeResults(results, sorted, reverse) return LazyCatToBrains(results) and then the script which actually issues the query can just issue a query directly to one catalogue or can query multiple catalogs and handle the results identically: if REQUEST.get('Occurrences')=='all': catalogs = context.multisite_catalog, catalog results = context.portal_indexer.queryMultipleCatalogs(query, *catalogs) else: #'this' results=catalog(query)

1 0

import of "aq_base" from "Acquisition" is unauthorized. You are not allowed to access 'aq_base' in this context
by KLEIN Stéphane 18 Mar '05

18 Mar '05

I've add "from Acquisition import aq_base" in python script in "portal_skins/custom/" When I test a page, I've this error : import of "aq_base" from "Acquisition" is unauthorized. You are not allowed to access 'aq_base' in this context Why this ? How to resolve it ? I search information about it in Zope Api and Google, I found nothing :( Thanks for your help, Stéphane

3 3

RE: [Zope] Double and Triple Posts
by Andy Yates 18 Mar '05

18 Mar '05

Thanks for the info! I wish I had posted this question months ago. When a user comes to the main page on my site 100+ values are loaded from a MySQL db and are stored in the SESSION object. These values are used to customize everything from text to images on the site. So, there are tons of reads and writes to the SESSION object going on all the time. The only problem I've seen is when I am sending email (multiple emails), processing credit cards (double,triple billing), and inserts into the db (duplicate inserts, key violations). 1. Is using the session object like this just a bad approach in general? 2. In the critical places where retries cause a problem can I just avoid reads and writes to session object? Will this prevent conflicts and the retries they cause? 3. Is there some other sessioning engine that can be used? Andy > -----Original Message----- > From: Chris McDonough [mailto:chrism@plope.com] > Sent: Tuesday, March 15, 2005 11:39 PM > To: Andy Yates > Cc: zope(a)zope.org > Subject: Re: [Zope] Double and Triple Posts > > This is almost certainly due to ZODB conflict errors, which are an > artifact of ZODB using an "optimistic concurrency" strategy for > concurrent updates. A lot of times people will come to the list and > say "My application is sending two or three mails when it should only > send one!" and that's a giveaway symptom of the code that sends the mail > experiencing a conflict error and retrying. > > Conflict errors are normal to a degree. If a ZODB conflict error > happens, Zope retries the request up to three times. The retry is > "invisible"; it happens within Zope itself. Most Zope apps can deal > with it because they use transactional data storages only, which get > rolled back, and then the request is replayed. It makes things slower, > but it eventually works. > > Conflict errors are common during sessioning usage (because sessions use > ZODB "under the hood" and people who use sessions may not have a lot of > experience writing "conflict-resistant" code), but they can be mitigated > somewhat. > > The best way to fix these is to change your code to maybe not do as much > work, to do its work faster, or to always use transactional data > storages instead of nontransactional ones (like syslog or MailHost, or > so forth). In the case of sessioning, using multiple frames/windows > that access the session at the same time is a common source of conflict > errors too. Someone "leaning" on their browser's reload button could > also cause conflict errors to occur. Also, if you store your session > data in a ZEO storage, conflict errors are likely to occur more > frequently due to increased latency when committing transactions. > > Unfortunately, there's no magic wand here and reviewing your code for > "conflict-resistance" borders on consulting work rather than simple > maillist help (it's particularly time-consuming and usually requires > actually running the application), so unless you ditch Zope sessions, > implement a more conflict resistant sessioning storage, or hire somebody > who is intimiately familiar with ZODB, you will likely need to learn > what causes these errors and how to mitigate them (or at least work > around them), maybe even by trial and error. > > It's worthwhile to know that any access to a Zope session actually has > the potential to *write* to the ZODB (even though it may appear you are > just reading), so heavy session usage is a prime candidate for > generating conflict errors. However, I've reduced them to an acceptable > level in my own projects, so I know it's possible. > > See also "Conflict Errors" within > http://www.plope.com/Books/2_7Edition/Sessions.stx .. this work by > Michael Dunstan might also be interesting to you.. > http://www.plope.com/Members/dunny/conflicts/view > > HTH, > > - C > > PS: please don't send HTML email to the list, it makes it hard for > people searching the maillist archives to read threads; thanks! > > On Tue, 2005-03-15 at 22:06, Andy Yates wrote: > > I have a form that posts to a python script which can take several > > seconds to run. Sometimes the script gets called 2 or 3 times in the > > same request. It is not easy to reproduce but it seems to happen at > > least once every 20 requests. This is happening on two different > > versions of Zope and in two completely different projects. > > > > > > > > Zope only logs a single request. However, I know it is getting called > > more than once because the script writes to syslog. > > > > > > > > I've packet sniffed the browser and it is only sending one post. It > > happens in IE and FireFox but seems to be easier to reproduce in IE. > > IE will let you click more than once on the submit button. When I > > click the submit button twice sometimes I get 3 or 4 runs of the > > script. I would expect to get only two. It seems to happen much more > > often when I do this. (FireFox will only let you press the submit > > button once) > > > > > > > > It seems to happen if the script reads and writes values to > > context.REQUEST.SESSION. When I take this out it seems to stop. (It > > can be hard to reproduce) > > > > > > > > To reproduce this I created a script that calls an external method to > > syslog a message. Then it reads and writes to > > context.REQUEST.SESSION. Then it calls another external method to > > sleep 15 seconds. Finally it prints a message and returns. > > > > > > > > > > > > I see this behavior on Zope-2.7.3 and Zope-2.6.1. > > > > > > > > > > > > context.zsyslog('Snooze test' ) > > > > > > > > foo = context.REQUEST.SESSION.get('foo', 1) > > > > if foo==1 : > > > > context.REQUEST.SESSION.set('foo', 0) > > > > > > > > context.REQUEST.SESSION.set('bar',123) > > > > > > > > context.Snooze(15) > > > > > > > > print "I'm awake now!" > > > > return printed > > > > > > > > Please let me know if this is happening anywhere else. > > > > > > > > Andy > > > > > > > > > > > > > > ______________________________________________________________________ > > _______________________________________________ > > Zope maillist - Zope(a)zope.org > > http://mail.zope.org/mailman/listinfo/zope > > ** No cross posts or HTML encoding! ** > > (Related lists - > > http://mail.zope.org/mailman/listinfo/zope-announce > > http://mail.zope.org/mailman/listinfo/zope-dev )

3 2

RE: [Zope] Hardware Platforms
by Jay Zeemer 17 Mar '05

17 Mar '05

Dieter Do you have any idea what your approximate unique sessions per day are, and what your peak usage is with this setup?? Just trying to get a handle on how well Zope will scale for our needs. Thanks for help ya'll have provided me with!! Jay -----Original Message----- From: Dieter Maurer [mailto:dieter@handshake.de] Sent: Wednesday, March 16, 2005 4:05 PM To: Jay Zeemer Cc: zope(a)zope.org Subject: Re: [Zope] Hardware Platforms Jay Zeemer wrote at 2005-3-16 08:33 -0500: >Currently we have been testing Zope 2.7.4 on Solaris 9. We plan to link >Zope to an MS SQL 2000 database as part of an existing process. The issues >we have been having is that we can not seem to get the system to handle more >then 50 concurrent connections. We have established this is more a problem >with the database connector then a problem with Zope. I was wondering if >anyone out there was using Solaris and MS SQL, and if so what drivers are >you using?? We use it without problems in the following configuration: 5 heavily used ZEO clients (Zope's) under Linux connecting to a single MySQL in the backend under Solaris 9.. We are using ZMySQLDB. -- Dieter

2 1

Unbuffered Python Scripts
by Nicholas Avenell 17 Mar '05

17 Mar '05

I have a script that generates a really big CSV file, which I'm doing as a python script inside Zope. Problem is, at the moment it takes huge amounts of memory and time to render, partially because the output is buffered before it's sent to the browser. Is there any way to stop this python script being buffered? Ideally I'd like each print statement to be flushed directly to the client, instead of returning "printed" at the end. -- Nicholas Avenell Technical Developer

3 2

RE: [Zope] Re: Security Bug -- To be fixed in Zope 2.7.5
by Jay Zeemer 17 Mar '05

17 Mar '05

Not to be a nag, but is there a date when this fix will be available?? Either in 2.7.5RC# or otherwise?? I admit I am not much of a Python programmer, as of yet, so most of what I am seeing is unfamiliar to me about where the fix is. Jay -----Original Message----- From: Tres Seaver [mailto:tseaver@zope.com] Sent: Thursday, March 10, 2005 3:27 PM To: zope(a)zope.org Cc: andreas.jung(a)haufe.de Subject: [Zope] Re: Security Bug -- To be fixed in Zope 2.7.5 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dieter Maurer wrote: | Malcolm Cleaton wrote at 2005-3-10 10:07 +0000: | |>... |> |>>It should not be necessary: |>> |>> A user should not be able to access any *protected* (!) object |>> outside the subhierarchy governed by the user folder |>> that authenticated the user. |>> |>>But maybe, we have a bug (and "aq_inContextOf" does not work |>>as expected). |> |>Yes, this shouldn't be necessary, and it looks like it's a bug. |> |>Looks to me like the bug is in User.py's allowed method. Quite simply, |>when it checks for the Authenticated role, it doesn't call |>self._check_context, |>so never attempts to detect and foil acquisition |>tricks. Unless I'm missing something, it should be a quick and easy fix. | | | You are right! Yep. The only hard part will be writing a decent unit test which exercises the bug: - -------------------- 8< ------------------ diff -u -r1.176.14.7 User.py - --- lib/python/AccessControl/User.py 25 Jan 2005 13:46:14 -0000 1.176.14.7 +++ lib/python/AccessControl/User.py 10 Mar 2005 20:26:53 -0000 @@ -182,7 +182,8 @@ ~ # role and user is not nobody ~ if 'Authenticated' in object_roles and ( ~ self.getUserName() != 'Anonymous User'): - - return 1 + if self._check_context(object): + return 1 ~ # Check for ancient role data up front, convert if found. ~ # This should almost never happen, and should probably be - -------------------- 8< ------------------ Tres. - -- =============================================================== Tres Seaver tseaver(a)zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCMK2AGqWXf00rNCgRAux+AJ0Zas9R/lUMc+Oot05jl5TNbunQLACeKBlt ZgoCjc6pOE8AjdSy6a7CUj8= =RLrC -----END PGP SIGNATURE----- _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

3 3

Zope, Plone, J2ee?
by Andres Valenciano 17 Mar '05

17 Mar '05

Hi, I am in the process of reviewing the design and features for version 2 of one of my company's internal applications ( for external clients, etc ). The first version was developed using Java with Struts and Tiles. I have now a "big question". There's a lot of features in Plone that I would like to implement ( events, calendars, preferences, cross-browser support ) basic stuff that I have to implement from scratch in a j2ee app. This application is a transactional one, inventory management, notifications, reports, etc. Is Zope or Plone able to work under this type of application? I am asking because Plone ( and Zope I think ) are referred as Content Management Systems and I can't find info about potentially heavy transactional systems working on top of this systems. The question is not if Zope or Plone are great because they are, but if they are a good ( better? ) platform for this other type of web applications, can it scale to support database/messaging transactions with potentially complex business logic for thousands of users? Thanks for your comments, And.

5 4

Transience errors in 2.7.2?
by Chris Withers 17 Mar '05

17 Mar '05

I'm getting the following on a Zope 2.7.2 server: Module Products.Transience.Transience, line 765, in new_or_existing Module Products.Transience.Transience, line 476, in get Module Products.Transience.Transience, line 280, in _move_item Module Products.Transience.Transience, line 694, in _gc KeyError: 1109697840 I thought all these errors were fixed? (or were they finally nailed in a version later than 2.7.2?) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

3 4

LocalFS
by Rakotomandimby (R12y) Mihamina 17 Mar '05

17 Mar '05

Hello! I want to use LocalFS to display some mrtg and webalizer charts on a Zope site. As you know those charts are refreshed, and I want some "real time" display of the chart, so that I cant upload one image one time and display it the HTML way. I need to take the chart from the File System and display it. I also have some other files I want to parse on the filesystem, so I really need LocalFS. But would you have any tutorial on using LocalFS, so that I could learn dealing with it? -- En mal de support technique? http://supports.etud-orleans.fr/ Infogerance de serveur dedie http://aspo.rktmb.org/activites/infogerance (En louant les services de l'ASPO vous luttez contre la fracture numerique) Etudiants de l'Universite d'Orleans: http://www.etud-orleans.fr/

3 2

Re: [Zope] newbie - Zope Error
by Stefan Elwesthal 17 Mar '05

17 Mar '05

I'll answer that myself! Not why it all of a sudden couldn't access a .gif picture, but I solved it somehow. Seemed to all be well again when I rebuilt my .html and switched header.gif ;) Now for some serious coding.. /Stefan -- ______________________________________________ Check out the latest SMS services @ http://www.linuxmail.org This allows you to send and receive SMS through your mailbox. Powered by Outblaze

1 0