July 2001 - Zope-Dev

Re: [Zope-dev] Re: ATTENTION: Zope trunk users
by Anthony Baxter 31 Jul '01

31 Jul '01

>>> Shane Hathaway wrote > On Monday 30 July 2001 17:22, Shane Hathaway wrote: > > So if you have an existing trunk checkout, cd to the "lib/python" > > subdirectory and execute the following commands: I found I just needed to checkout a new copy entirely - things just didn't play properly. Anthony

1 0

RE: [Zope-dev] Join mix and match queries in Zope (was: Catalog r esult brains not extendable)
by Jay, Dylan 31 Jul '01

31 Jul '01

> -----Original Message----- > From: Evan Simpson [mailto:evan@4-am.com] > Sent: Tuesday, 31 July 2001 12:25 PM > To: Chris McDonough; Jay, Dylan > Cc: zope-dev(a)zope.org > Subject: Re: [Zope-dev] Catalog result brains not extendable > > > > "Jay, Dylan" wrote: > > > I'm trying to get some results from a catalog and add meta_data to > it. > > In $ZOPE/lib/python/Shared/DC/ZRDB/Results.py find the "class r" > definition, and add one line to it, like this: > > class r(Record, Implicit, brains, zbrains): > _guarded_writes = 1 > > Now you can assign to attributes of result records. Cool, is there a good reason this can't be activated by default for Catalog results and SQLMethod results? Alternatively some mechanism to plug in your own brains into a Catalog query. I'm trying to think of ways to generalize solutions to my problem since I think it can be applicable to others. The problem is joining query results with other dynamic data in my case data specific to the user making the query. In particular this user specific data specifies the sort order of the combined result. Perhaps drop-in indexes can provide a mechanizm? If you had an catalog index which had many indexes, one for each user, then the result would be the same. However in my case I don't want all that data recorded at one time. I generate the index dynamically (which is expensive) and then cache it so that the frequent users have their indexes available. So to generalize this you could have a drop-in index that can generate the index on the fly based on current environmental conditions such as the logged in user. This would be done in such a way as to allow a caching manager to sit in between. I don't know enough about the catalog machinery to know if this is feasible, anyone? or if it would be useful to others? Another possible solution is to allow the efficient joins of two different queries. Let say I make a Catalog like result set from my user specific index. I also get a result set by querying the catalog. I then pipe both into a joiner tool that efficiently (and lazily) gives me a result of the two joined on a specified attributes. This could be generalized to SQLMethods as well. Then relational data can be joined with catalog data or catalog with catalog or relational for two different sources joined. This proposal is similar to one I saw about putting SQL right inside the zope environment. Using the Gadfly code or similar and doing SQL functionality on zope objects etc. You could extend the ZSearchable interface idea to more things and make it work with an SQL interpretter inside zope. Folders could be ZSearchable as could Find. Arbitrary scripts could be and thus participate at an equal level. You could have your SQL in an SQLQuery object. Your Find search in a FindQuery object that is like a stored search. The sql references it by name. So later when you want it to be more efficient you replace it with a CatalogQuery or perhaps just use a standard cache manager to do some caching. Am I getting to proposals that people see of use to more people than me?

1 0

ATTENTION: Zope trunk users
by Shane Hathaway 31 Jul '01

31 Jul '01

This is for everyone using a trunk CVS checkout of Zope. (Until you do this, Zope won't start.) To facilitate the StandaloneZODB effort and in the name of general tidyness, we're changing four modules into packages. Unfortunately, our current CVS setup does not allow these packages to be added automatically to existing checkouts. So if you have an existing trunk checkout, cd to the "lib/python" subdirectory and execute the following commands: cvs co -d ThreadedAsync Packages/ThreadedAsync cvs co -d zLOG Packages/zLOG cvs co -d zdaemon Packages/zdaemon cvs co -d Persistence Packages/Persistence cvs -q up -dP That should get you fully up to date. Shane

3 3

Catalog result brains not extendable
by Jay, Dylan 31 Jul '01

31 Jul '01

I'm trying to get some results from a catalog and add meta_data to it. It seems the only way I can do this using python scripts is by turning the each result object into a dictionary and then adding to that. This is due to the fact that you can't do a setattr on anything using a python script. Why not? Obviously for security reasons but surely there must be a less restrictive alternative than a blanket ban on using attributes on classes?

4 4

Re: [Zope-dev] BTrees and ConflictErrors.
by Anthony Baxter 30 Jul '01

30 Jul '01

>>> Chris McDonough wrote > Well, if the version of 2.3 you're using indeed does have a > conflict-resolving storage, it's likely that the conflict resolution > code is being called (you may be able to put a print in there just to be > sure). But not all conflicts can be resolved. In this case, it's adding two session objects to a session data container - I'd expect this would be relatively trivial to handle. (but I've been wrong before :) -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

1 0

Security Wierdness
by Michael R. Bernstein 30 Jul '01

30 Jul '01

Hi everyone, Ok, I'm developing a python Product and have gotten to the stage where I am adding security declarations. I understand the default policy (once you've added a ClassSecurityInfo instance to the class and called InitializeClass(yourClass) ) should be to deny access to methods that don't have any security declarations. This doesn't seem to be the case. Even though I have *not* added security.setDefaultAccess("allow"), access to unprotected methods (both regular methods and HTMLFile methods) is being allowed rather than denied. Protecting methods with a permission works as it should. Can anyone help diagnose this? Michael Bernstein.

3 7

RE: Catalog Query Feature Request,was: RE: [Zope-dev] An idea for Un iqueValuesFor
by sean.upton＠uniontrib.com 30 Jul '01

30 Jul '01

For what it's worth, here is the code I am using to parse the query and add the wildcards; it seems to be non-intrusive towards all ops, including (), and it's been moderately tested. It might be slightly inefficient, though... Feel free to borrow and/or improve on this if it is useful. Sean def queryExtender(self, query): """ Takes, as input, query for Text index of ZCatalog, and makes it more intelligent by parsing it and rewriting it to include wildcards at the end of words so that we can search sub-words; in other words, a search for something like "engineer" should yield results for "*engineer*" so that terms like "engineers" and "engineering" also are considered matches. Obviously, we have to be careful not to incorrectly parse the query, and we don't want to mess with words that already have wildcards at the end, because you don't want to end up with something like "*engineer**" """ ### Define Character Patterns to Strip Out and Split Upon everythingButSearchTerms = '[^A-Za-z0-9*]+' #Regex Pattern ### Create the word list result = re.split(everythingButSearchTerms, query) ### Get rid of empty string elements in the word list try: for i in range(result.count('')): result.remove('') except: pass ### Get rid of boolean operators booleanops = '^([Aa][Nn][Dd])|([Oo][Rr])|([Aa][Nn][Dd][Nn][Oo][Tt])|([Nn][Ee][Aa][Rr])$' i=0 #count variable, used for indexing for item in result: if re.search(booleanops, item): result.pop(i) i = i + 1 ### Now, result is a list of just the words that are ### meaningful to the search, but we need to eliminate ### any entries that have wildcards in them, because ### they are likely more specific than our rewrite here asteriskinterm = '(^[*])|([*]$)$' #asterisk at start or end of term i=0 #count variable, used for indexing for item in result: if re.search(asteriskinterm, item): result.pop(i) i = i + 1 ### Now, the list of words in the query we need to modify is ### final, so we can start modifying the queries, one word ### at a time... for item in result: query = re.sub(item, '*'+item+'*', query, count=1) return query -----Original Message----- From: Casey Duncan [mailto:cduncan@kaivo.com] Sent: Monday, July 30, 2001 10:36 AM To: sean.upton(a)uniontrib.com; zope-dev(a)zope.org Subject: Re: Catalog Query Feature Request,was: RE: [Zope-dev] An idea for Un iqueValuesFor sean.upton(a)uniontrib.com wrote: > > I could definitely see the value of a unique-values query into ZCatalog, > especially for creating things using <dtml-tree> using keywords, etc... I'm wondering the best way to implement this on the API side, since it would change the output from catalog results to just attribute values. Any thoughts? > > On a slightly related (well, not really) note, CatalogQuery looks like it > would solve a lot of problems I have had with a very Catalog-intensive > application. One thought I had - I might suggest the possibility of adding > a fuzzy matching operator to CatalogQuery that performs the function of > wrapping wildcard searches on search terms for Text Indexes, supposing the > Catalog is using a globbing vocabulary: > > ~= as an operator would mean an approximate (substring) match > > So a search for 'title ~= "engineer"' would perform a search for > '*engineer*' and return results containing words like engineer, engineers, > engineering, etc. That sounds like a good idea. Would a simple split/join work, something like: ops = ('and', 'or') words = query_string.lower().split() for word in words: if word not in ops: word = '*%s*' % word query_string = words.join() I can look at adding this capability > > Right now, I attempt to safely rewrite REQUEST['someFieldThatIamSearching'] > with a Python class method that uses a zillion re.sub() calls to wrap search > terms in * characters; I wonder if there is a way to alternately implement > something like this at a lower level, perhaps in CatalogQuery; I get the > feeling it would be quicker and much more simple. > > If something like that were implemented as well as some equivalent to > sort_on, I'd stop pulling my hair out with traditional workarounds and > definitely switch all my stuff to use CatalogQuery instead... Yeah, I definitely want to add a sort_on capability. I think I will implement it as an optional argument, like it is for ZCatalog, rather than as part of the query string, at least for now. > > Thoughts? > > Sean > > -----Original Message----- > From: Casey Duncan [mailto:cduncan@kaivo.com] > Sent: Monday, July 30, 2001 8:20 AM > To: Chris Withers > Cc: zope-dev(a)zope.org; Anthony Baxter > Subject: Re: [Zope-dev] An idea for UniqueValuesFor > > Chris Withers wrote: > > > > Casey Duncan wrote: > > > > > > possibly, yes. I'll look to add this to my CatalogQuery product. I > > > believe the btrees can be pressed into service here... > > > > Hadn't heard of this CatalogQuery product... where can I find out more? > > > > I think I may have been about to develop something similar, so maybe we > can help > > each otehr out? > > > > cheers, > > > > Chris > > > > http://www.zope.org/Members/Kaivo/CatalogQuery > > This is my first stab at this. I forsee a much more general query > mechanism in the future, but this works better than the stock stuff (for > me) and it works today! > > Let me know what your ideas are... > > -- > | Casey Duncan > | Kaivo, Inc. > | cduncan(a)kaivo.com > `------------------> > > _______________________________________________ > Zope-Dev maillist - Zope-Dev(a)zope.org > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) -- | Casey Duncan | Kaivo, Inc. | cduncan(a)kaivo.com `------------------> _______________________________________________ Zope-Dev maillist - Zope-Dev(a)zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )

1 0

SV: [Zope-dev] SOAP support: request for comments
by Magnus Heino (Rivermen) 30 Jul '01

30 Jul '01

> Look at what Shane's been up to on the NR-branch in CVS. Are there any docs about this? I didn't find anything on dev.zope.org /Magnus

4 3

Re: [Zope-dev] BTrees and ConflictErrors.
by Anthony Baxter 30 Jul '01

30 Jul '01

>>> Anthony Baxter wrote > I thought the newer BTrees code was meant to deal with conflicts from > two entries being added at the same time? > I'm using CoreSessionTracking 0.8 with an external data source on a > bsddb3Storage.Minimal storage mounted via ZEO (using the current Zope2.3 > code). If two sessions are created around the same time, I'm seeing > ConflictErrors - there doesn't seem to be any attempt to invoke the > resolveConflict code. Ok, things that I've found: Minimal doesn't even try to implement Conflict resolution - I've switched to 'Full'. To no real avail, unfortunately - it's still breaking when I hammer it (10 clients all creating one new session per second). I've tried putting the SessionData object in a normal Data.fs - still no joy. I've also tried using CST's internal data container (RAM based). The ConflictErrors keep on flying. About the only thing I haven't tried yet is to try Zope2.4 - this is something of a pain, though. Has anyone got any other ideas? Am I just completely out of luck here? Do people have other suggestions for containers that can take a large number of new object create/deletes? Thanks, Anthony

3 3

BTrees and ConflictErrors.
by Anthony Baxter 30 Jul '01

30 Jul '01

I thought the newer BTrees code was meant to deal with conflicts from two entries being added at the same time? I'm using CoreSessionTracking 0.8 with an external data source on a bsddb3Storage.Minimal storage mounted via ZEO (using the current Zope2.3 code). If two sessions are created around the same time, I'm seeing ConflictErrors - there doesn't seem to be any attempt to invoke the resolveConflict code. Any ideas what I'm missing here? Anthony

2 1