June 2000 - Zope-Dev

Problem with XML in Zope 220b1
by Christian Wittern 13 Jun '00

13 Jun '00

Hello Fellow Zopers, I just downloaded Zope 220b1 and got it running. Now I run into a nasty problem: I have a lot of Chinese XML-files stored in Zope. The internal encoding is UTF-8. Everything was fine in 214, 216 and 220a1. Now with 220b1, some characters are (apparently?) randomly turned into lt;, gt; and the like. Now this looks like some unwanted HTML escaping, but the leading '&' is missing and the characters are definitely all in the range greater 127 (this is a property of UTF8), so there is no direct relationship to the codepoints of >, < and co. Any ideas what could have gone wrong here? All the best, Christian Dr. Christian Wittern Chung-Hwa Institute of Buddhist Studies 276, Kuang Ming Road, Peitou 112 Taipei, TAIWAN Tel. +886-2-2892-6111#65, Email chris(a)ccbs.ntu.edu.tw

1 0

Re: advice on MountedFileStorage changes.
by Anthony Baxter 13 Jun '00

13 Jun '00

>>> Shane Hathaway wrote > I looked some more at ZClasses and such today, hoping to solve the > problem with mounted databases. Guess what! It turned out to be a > simple matter. Excellent! I've added support in CVS, and I'll do a new release this evening. > The latest addition to Mount.py provides the ability for a MountPoint > to get its ZClass definitions from the root database rather than the > mounted database. Anthony, I think you'll want to make > "classDefsFromRoot" a checkbox available when adding a new mount point. Done. Cool, it looks like it even handles recursive mounts. > Depending on the application, this can be the more sensible (and more > manageable) approach. For example, you may want to have a large > collection of little FileStorages, all using the same ZClass > definitions. You certainly wouldn't want to make a copy of all > ZClasses in each FileStorage. Exactly. For me, one of the big things about the MountedFileStorage is the ability to split out the data for various sections of the tree - I still want to be able to make ZClass instances, but I don't want to have to manage copying all products all over the place. > In doing this I realized, and Jim suggested, that there are other > situations where you'd really want ZClass definitions to come from the > mounted database. This is a larger issue. We will first need to > provide a way to manage the databases. Then there will be a need to > load/unload products dynamically. I don't think we want to try to > finish that for 2.2.0. The next MountedFS release will have a teeny tiny Control_Panel like thing in the root of the mountpoint. I'll try putting a Products folder in it. Anthony

1 0

ZCatalog text index search bugs?
by R. David Murray 12 Jun '00

12 Jun '00

I am very confused. I'm looking at the SearchIndex source under 2.1.4 (2.1.6 seems to be the same). In Lexicon.py the 'query' method defines the default_operator to be 'or'. I can't see that TextIndex overrides this when it calls it. But the response to PR 1141 (against 2.1.6) in the collector says: The TextIndex search does an AND, not an OR, of the search words: if you ask it to find "foo bar", it returns only objects matching *both* "foo" and "bar", rather than object matching *either* "foo" or "bar" (which Jason expected). Indeed, if you do a search that includes a word that is not on an item, the item is not returned. So how is that working? A possible answer is: if you do a search like 'something or somethingelse', this *also* does not return the object if one of those words is not on the object. So is 'or' searching broken? Note that if you do a search like "something or with", this returns the object, "with" being a stop word. So does "something with". On the other hand, "something and with" does *not* return the object. So I think 'or' searching is broken, and that text indexes being a default 'and' search is just an accident <grin>. Following up on the 'something and with', though: Since "with" is a stop word, it can never be on the object. Since the user entering search words into the search form doesn't know what the list of stop words is, this stikes me as broken behavior. Anyone disagree? I also have a problem with a word such as "T-shirt". If I search on "T-shirt", my object that has that word in its text index does not show up. The splitter should be breaking that into "t" and "shirt", right? Is the problem that single letters are discarded by the Splitter, therefore T is like a stop word (but it isn't in the stopword table), therefore the implicit 'and' search(*) fails? To corroborate this, a search for "something t" finds that record, but "something and t" does not. This can't be the whole answer, though, since searching on just 'shirt' does *not* return the object. (*) I recall reading that the 'near' operator, which is used if the splitter breaks up a word in the search string, is not really supported and that the 'and' operator is used instead.) I can't tell yet if this bug is (these bugs are?) fixed in 2.2.0b1 since I can't see the source release yet. Looking at the a1 source, things have moved around a bit. But I see that "default_operator" is still set to 'or', so I suspect these bugs may remain... If I can reproduce this in 2.2.0b I'll file it in the collector. --RDM

1 0

More useful CatalogAware class?
by Michel Pelletier 12 Jun '00

12 Jun '00

Amos and I and the guys at Activestate were discussing cataloging at a recent training sessions and we came up with the following possible successor to CatalogAware. The idea is that a CatalogAware object looks for two methods that return sequecnes of Catalogs that many be interested in this object. The first is class defined method 'getClassCatalogs', the second is an instance defined method 'getLocationCatalogs' that the instance can get from its classes, its instance dict, or acquisition. The idea is that you provide methods that define the policy for finding Catalogs. The default class defined 'getClassCatalogs' is identical behavior to the old CatalogAware, but you can and probably will override it. Thoughts? -Michel """ This is a candidate successor to the CatalogAware class. """ class CatalogAware: meta_type = 'CatalogAware' default_catalog = 'Catalog' def getClassCatalogs(self): """ You want to over ride this method if you don't like the default policy of acquire a Catalog named 'Catalog' """ if hasattr(self, self.default_catalog): return [getattr(self, self.default_catalog)] def manage_afterAdd(self, item, container): self.catalog_object() for object in self.objectValues(): try: s=object._p_changed except: s=0 object.manage_afterAdd(item, container) if s is None: object._p_deactivate() def manage_afterClone(self, item): self.catalog_object() for object in self.objectValues(): try: s=object._p_changed except: s=0 object.manage_afterClone(item) if s is None: object._p_deactivate() def manage_beforeDelete(self, item, container): self.uncatalog_object() for object in self.objectValues(): try: s=object._p_changed except: s=0 object.manage_beforeDelete(item, container) if s is None: object._p_deactivate() def get_catalogs(self): cats = [] # first, try and call a class defined method to get # catalogs... we don't want to acquire... if hasattr(self.aq_base, getClassCatalogs): for cat in self.getClassCatalogs(): if cat not in cats: cats.append(cat) if hasattr(self, getLocationCatalogs): # next, try and call a method that can be class defined, # instance defined, or acquired. for cat in self.getLocationCatalogs(): if cat not in cats: cats.append(cat) return cats def catalog_object(self): """ Call this to catalog yourself """ for catalog in self.get_catalogs(): catalog.catalog_object(self, self.getPhysicalPath()) def uncatalog_object(self): """ Call this to uncatalog yourself """ for catalog in self.get_catalogs(): catalog.uncatalog_object(self.getPhysicalPath()) def recatalog_object(self): self.uncatalog_object() self.catalog_object()

2 1

Proposed interface for application-level conflict resolution in ZODB
by Jim Fulton 12 Jun '00

12 Jun '00

Zope takes a pretty optimistic approach to dealing with conflicting database changes. Sometimes, there may be "hot spots" in an application that get a lot of simultaneous writes. These hot spots can often be designed away. An alternative is to provide application-level logic for sorting out the changes made by conflicting writes. The proposed interface at: http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResolution provides a way to provide this application logic. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

1 0

advice on MountedFileStorage changes.
by Anthony Baxter 12 Jun '00

12 Jun '00

I've nearly completed the code to make MountedFileStorage able to create new FileStorages on demand, _however_... I'm hitting a bit of a problem with the Product initialisation. Right now I'm just calling OFS.Application.initialize(), which goes through and re-imports all the products, and adds them to the global meta_types list. This is bad - it means it's slow as hell reimporting everything, and everything gets listed twice. It's fine after a restart, but that's not much help. I can't just skip product initialisation, as it's kinda needed to create objects in the sub-zodb. So, approaches I'm considering: passing a flag through initialize() saying 'don't put products in the meta_types' (seems hacky). subclassing OFS.Application.Application, and copying the method except for the last line that alters the meta_types (fragile - have to track changes to the main body of code). trying something else entirely, such as copying the products from the main products folder, and installing them that way... ? Any ideas appreciated. (There's an ongoing issue, too - there will need to be some sort of way to copy any new/changed Products from the main ZODB to the children...) Anthony

2 1

Testing external methods with arguments [PATCH]
by Gisle Aas 12 Jun '00

12 Jun '00

While writing on the perl external method product I back ported its "Try It!" support to also work for python external methods that take arguments. The change is fairly simple. Clicking the "Try it!" tab takes you to a screen where you get a chance to fill out the arguments for the method before you actually call it. If the function takes no arguments, then things work the same as they do today. I introduced a "call" alias for "__call__" in order to be able to actaully call the method object from the DTML method. Is there a better way? Regards, Gisle Index: lib/python/Products/ExternalMethod//ExternalMethod.py =================================================================== RCS file: /local/src/CVSROOT/Zope2/lib/python/Products/ExternalMethod/ExternalMethod.py,v retrieving revision 1.1.1.1 diff -u -p -u -r1.1.1.1 ExternalMethod.py --- lib/python/Products/ExternalMethod//ExternalMethod.py 2000/06/07 20:38:02 1.1.1.1 +++ lib/python/Products/ExternalMethod//ExternalMethod.py 2000/06/12 13:41:22 @@ -160,7 +160,7 @@ class ExternalMethod(OFS.SimpleItem.Item ( {'label':'Properties', 'action':'manage_main', 'help':('ExternalMethod','External-Method_Properties.dtml')}, - {'label':'Try It', 'action':'', + {'label':'Try It', 'action':'manage_try', 'help':('ExternalMethod','External-Method_Try-It.dtml')}, ) +OFS.SimpleItem.Item.manage_options @@ -205,6 +205,8 @@ class ExternalMethod(OFS.SimpleItem.Item action =REQUEST['URL1']+'/manage_main', target ='manage_main') + manage_try=HTMLFile('methodTry', globals()) + def getFunction(self, check=0, reload=0): f=getObject(self._module, self._function, reload) @@ -271,7 +273,8 @@ class ExternalMethod(OFS.SimpleItem.Item raise TypeError, v, tb finally: tb=None - + + call = __call__ def function(self): return self._function def module(self): return self._module The 'methodTry.dtml' file looks like this: ------------------------------------------------------------------>8-------- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd"> <HTML lang="en"> <HEAD> <TITLE>Edit <dtml-var title_or_id></TITLE> </HEAD> <BODY BGCOLOR="#FFFFFF" LINK="#000099" VLINK="#555555" ALINK="#77003B"> <dtml-var manage_tabs> <FORM ACTION="<dtml-var URL1>" METHOD="GET"> <h2>Calling <dtml-var title_or_id> <dtml-with func_code> <dtml-if co_argcount> with arguments: </h2> <table border=0> <dtml-in co_varnames> <dtml-if "_['sequence-index'] < co_argcount"> <tr><td align=right><b><dtml-var sequence-item html_quote></b>:</td> <td> <dtml-if "_['sequence-item'] in ('self', 'REQUEST')"> <i>Provided by Zope's ZPublisher internally</i> <dtml-else> <input name="<dtml-var sequence-item html_quote>" size=30> </dtml-if> </td> </tr> </dtml-if> </dtml-in> </table> <dtml-else> with no arguments!</h2> <dtml-comment> <dtml-raise type="Redirect"><dtml-var URL1></dtml-raise> </dtml-comment> <dtml-return call> </dtml-if> </dtml-with> <INPUT TYPE="SUBMIT" VALUE="Execute"></TD> </FORM> </BODY> </HTML> ------------------------------------------------------------------>8--------

1 0

broken products after update
by Doug Wyatt 11 Jun '00

11 Jun '00

Hi, I just updated from Zope 2.0.1 to 2.1.6. All of the Python-based products I was using previously have been migrated successfully. But I have a number of products that I created with ZClasses. I can examine these products in the control panel, and everything is there and looks fine, but all of these classes' object instances are broken. I get this message when clicking on one in the management interface: "This object is broken because the unknown product that created it is no longer installed or is installed incorrectly. Please contact your product vendor for assistance." "Note that the data associated with this product has not been lost, and will be accessible again if the product is reinstalled." I'll keep digging for documentation I may have missed, but I'd really appreciate some help. Doug (would appreciate copies by private email as I am on the digest) -- Doug Wyatt doug(a)sonosphere.com http://www.sonosphere.com/

2 1

SOLVED: broken products after update
by Doug Wyatt 11 Jun '00

11 Jun '00

OK, I finally dug far enough through the list archives to find a suggestion that worked. My old Zope had lots of products installed. I wasn't using a lot of them, so I didn't install them into the new Zope. To solve the problem, I had to delete from the database the products that were no longer installed. To be able to delete ZDiscussions (I think it was), I had to actually copy Products/ZDBase into the new installation. Then I could delete the products that depended on it. Now that the database no longer refers to products which are no longer installed, all is well. I seem to remember a comment somewhere, saying that this was on DC's radar as something to fix. Please :-) Thanks, Doug - - - - - Hi, I just updated from Zope 2.0.1 to 2.1.6. All of the Python-based products I was using previously have been migrated successfully. But I have a number of products that I created with ZClasses. I can examine these products in the control panel, and everything is there and looks fine, but all of these classes' object instances are broken. I get this message when clicking on one in the management interface: "This object is broken because the unknown product that created it is no longer installed or is installed incorrectly. Please contact your product vendor for assistance." "Note that the data associated with this product has not been lost, and will be accessible again if the product is reinstalled." I'll keep digging for documentation I may have missed, but I'd really appreciate some help. Doug (would appreciate copies by private email as I am on the digest) -- Doug Wyatt doug(a)sonosphere.com http://www.sonosphere.com/

1 0

Re: [Zope-dev] advice on MountedFileStorage changes.
by Anthony Baxter 11 Jun '00

11 Jun '00

>>> Anthony Baxter wrote > > I've nearly completed the code to make MountedFileStorage able to > create new FileStorages on demand, _however_... > > I'm hitting a bit of a problem with the Product initialisation. Right > now I'm just calling OFS.Application.initialize(), which goes through > and re-imports all the products, and adds them to the global meta_types > list. This is bad - it means it's slow as hell reimporting everything, > and everything gets listed twice. It's fine after a restart, but that's > not much help. Ok, I've gone with the 'subclass Application' approach, and made a couple of slight changes to the core code. Now, stuff won't be added to global registries like Products.meta_types if the application doesn't have isTopLevelPrincipiaApplicationObject set to true. It seems to work well, patch follows - any comments? Anthony. This patch (against current CVS, 2000-06-11) stops the addition of a new MountedFileStorage adding all the products to the master meta_types list. It's not _essential_, but without it, you'll find you need to restart Zope to get the 'Add' list back to sanity. Index: OFS/Application.py =================================================================== RCS file: /cvs-repository/Zope2/lib/python/OFS/Application.py,v retrieving revision 1.128 diff -u -r1.128 Application.py --- Application.py 2000/06/09 23:52:45 1.128 +++ Application.py 2000/06/11 07:00:30 @@ -511,7 +511,8 @@ error=sys.exc_info()) get_transaction().abort() - Products.meta_types=Products.meta_types+tuple(meta_types) + if app.isTopLevelPrincipiaApplicationObject: + Products.meta_types=Products.meta_types+tuple(meta_types) Globals.default__class_init__(Folder) Index: App/ProductContext.py =================================================================== RCS file: /cvs-repository/Zope2/lib/python/App/ProductContext.py,v retrieving revision 1.22 diff -u -r1.22 ProductContext.py --- ProductContext.py 2000/06/09 01:22:13 1.22 +++ ProductContext.py 2000/06/11 07:00:30 @@ -202,12 +202,13 @@ if not hasattr(pack, '_m'): pack._m={} m=pack._m - Products.meta_types=Products.meta_types+( - { 'name': meta_type or instance_class.meta_type, - 'action': ('manage_addProduct/%s/%s' % (pid, name)), - 'product': pid, - 'permission': permission, - },) + if app.isTopLevelPrincipiaApplicationObject: + Products.meta_types=Products.meta_types+( + { 'name': meta_type or instance_class.meta_type, + 'action': ('manage_addProduct/%s/%s' % (pid, name)), + 'product': pid, + 'permission': permission, + },) m[name]=initial m[name+'__roles__']=pr

1 0