- Zope-Dev - Zope lists

ZCatalog glitches
by Dieter Maurer 20 Aug '00

20 Aug '00

The Zope 2.1.6 ZCatalog was not bad but had a set of glitches. The Zope 2.2.1b1 ZCatalog (CVS from 2000-8-19) is much better but still has a set of glitches: * UnTextIndex(349) breaks, if the lexicon lookup returns the empty tuple --> "index out of range". * UnTextIndex(349) only looks at the first wordindex returned by the lexicon lookup -- surely not what one expects, if the lexicon is globbing * GlobbingLexicon(178) breaks, if the second pattern character is a wildcard. * GlobbingLexicon.GlobbingLexicon.translate anchors the resulting pattern at the end but not the start. This implies that a word matches the pattern (as defined by lexicon), if it has a suffix that matches (as defined by "re.match"). This is quite a strange semantics. Attached is a patch, fixing these glitches. Dieter --- lib/python/SearchIndex/:GlobbingLexicon.py Sat Mar 25 18:42:32 2000 +++ lib/python/SearchIndex/GlobbingLexicon.py Sun Aug 20 12:46:42 2000 @@ -164,28 +164,42 @@ def get(self, pattern): """ Query the lexicon for words matching a pattern. + DM: Note: the complete word must mach *pattern*. + Formerly, a word was matched when it ended + in a match of *pattern*. This, probably, was + a bug. """ + # import pdb; pdb.set_trace() + wc_set = [self.multi_wc, self.single_wc] digrams = [] + no_wc= 1 # DM: optimization for i in range(len(pattern)): if pattern[i] in wc_set: + no_wc= 0 continue if i == 0: digrams.insert(i, (self.eow + pattern[i]) ) - digrams.append((pattern[i] + pattern[i+1])) - else: - try: - if pattern[i+1] not in wc_set: - digrams.append( pattern[i] + pattern[i+1] ) - - except IndexError: - digrams.append( (pattern[i] + self.eow) ) + try: + if pattern[i+1] not in wc_set: + digrams.append( pattern[i] + pattern[i+1] ) + else: no_wc= 0 + + except IndexError: + digrams.append( (pattern[i] + self.eow) ) + # DM: optimization + if no_wc and self.anchored: + # we have no wildcard; therefore we may look up the + # word directly + if self._lexicon.has_key(pattern): return (self._lexicon[pattern],) + else: return () + ## now get all of the intsets that contain the result digrams result = None @@ -248,6 +262,10 @@ return Splitter(astring) + # DM: optimization + # anchored: true means, the pattern must match the complete word + anchored= 1 + def translate(self, pat): """Translate a PATTERN to a regular expression. @@ -255,7 +273,7 @@ """ i, n = 0, len(pat) - res = '' + res = self.anchored and '^' or '' while i < n: c = pat[i] i = i+1 @@ -265,7 +283,7 @@ res = res + '.' else: res = res + re.escape(c) - return res + "$" + return self.anchored and res + "$" or res --- lib/python/SearchIndex/:UnTextIndex.py Sat Aug 19 19:27:01 2000 +++ lib/python/SearchIndex/UnTextIndex.py Sun Aug 20 12:43:22 2000 @@ -346,10 +346,15 @@ if len(src) == 1: src=src[0] if src[:1]=='"' and src[-1:]=='"': return self[src] - r = self._index.get(self.getLexicon(self._lexicon).get(src)[0], - None) - if r is None: r = {} - return ResultList(r, (src,), self) + r= None + for wordindex in self.getLexicon(self._lexicon).get(src): + rr = self._index.get(wordindex, None) + if rr is not None: + rr= ResultList(rr, (src,), self) + if r is None: r= rr + else: r= r.__or__(rr) + if r is None: r = ResultList({}, (src,), self) + return r r = None for word in src:

1 0

Re: [Zope-dev] patch for &dtml-foo:bar:baz;
by Jon Franz 19 Aug '00

19 Aug '00

This may, or may not, break the alternate syntax patch/download @ http://www.zope.org/Members/Coventry/dtml_shortcut I'll investigate and find out, and release a new dtml_shortcut if needbe. (I specifically used a colon since it is valid xml) What about using a colon as the separator, but explicitly give the tag the name dtml-root? <dtml-root foo:bar:childofbar> having a whole tag of type ':' seems weird (in addition to possibly breaking current patches and such ;) if we didn't want the root to be the default we could use something like <dtml-trans [path]:[path]> with something like _root or __root being set aside as a special name for starting at the root (otherwise it would start in the current directory)...? >>> This patch changes lib/python/DocumentTemplate/DT_HTML.py so that you can use paths for traversal in DTML entity syntax. The delimiter is ':', as it is valid as part of an XML entity, but not valid as part of a Zope object id. You can use &dtml-:foo:bar; to ensure that traversal occurs from the root object. This patch is against Zope 2.2.1b1. It relies on the restrictedTraverse() api. A 2.1.x version would be possible, but would be rather messy. <<<

2 1

DCOracle w/ 8.0.5 and HP-UX 11
by Ron_Wood＠capgroup.com 18 Aug '00

18 Aug '00

Has anyone gotten this to work. I can get everything to compile but get a bus error when running DCOracle_test.py when importing oci_. I have python compiled with threads and used the -D_POSIX_C_SOURCE=199506L (-D_REENTRANT) flag. Any help would be greatly appreciated, Ron Wood

1 0

ZPatterns and "Strategies & Patterns etc...."
by Steve Spicklemire 18 Aug '00

18 Aug '00

Hi Folks, Well.. I never heard any comments about my last question.. so I thought I'd try to frame it differently. I just read Coad's book on Object Models etc.. and I think I pretty well 'grok' it.. at least enough to be mildly dangerous. ;-) I'm implementing a project with ZPatterns and I would like to keep a balance between flexibility and sanity (and of course performance). I've been using 'raw Specialists' to keep the number of new classes down to a bearable number, but I'm beginnnig to thing that Zope would behave itself better if I were to create Product level sub-classes of Specialist instead. I guess I'd just like to 'hear' what folks think about these different strategies: at one end of the spectrum: 1) Use ZClasses and 'out of the box' specialists for 90% of the application. Make ZClass and only where python is *required*, use external methods. at the other end of the spectrum: 10) Use full Python products, possibly subclassed from ZPatterns classes (DataSkin, Specialist etc....) and never use ZClasses at all... and there is the middle ground: 5) Use python products for low level logic. Create overriding methods that can be 'dropped in' at the admin interface level, or in a subclass, later to change the behavior (e.g., retrieveItem ). Make ZClass sub- classes so that userinterface can be easily modified later, and so that persistent propertysheets are easy to add... There are problems with each end of this spectrum... 1) There are still limitations on what you can easily do outside of python. PythonMethods are great.... but they have strict security limitations that make them cumbersome in some cases. When you want to call a method of an object, that is really a 'DTML/Python Method' you need to pass along context so you end up with a lot of: <dtml-with FooManager> <dtml-let theResult="aMethodOfTheFooManager(.... )"> </dtml-let> </dtml-with> This seems like a lot of monkey-business just to call a method. Also.. nested withs can generate problems with namespace pollution.. and unfortunately performance. 5) Same problems as above when invoking methods that are implemented as DTML/Python Methods.. otherwise.. not too bad.. this is where I'm living at the moment. All my DataSkin classes are structured as: DataSkin --> myBaseClass -> zSubClassableWrapper -> ZClass My Specialists are just plain old Secialists... (here's the problem.. should they be Python products so that they can have plain Python methods? 10) Everything is defined in Products... harder to integrate into other apps since it's hard to override/change methods without modifying source code on the file system. I can't believe everyone is doing it this way.. but then I con't believe that folks are pushing and popping namespaces all over the place like I seem to in my app.... Anyway.. thoughts are appreciated. ;-) thanks, -steve

2 1

Re: [Zope-dev] why is manage_addZSQLMethod unavailable ?
by Rik Hoekstra 18 Aug '00

18 Aug '00

>Why does this work > ><dtml-call "_['testadd'].manage_addDTMLMethod('MethodId', 'Method >Title','method text')"> > > >While this does not: > ><dtml-call "_['testadd'].manage_addZSQLMethod('ZSQLID', 'ZSQL Title', >'DB','','select * from data')"> > >Error Type: AttributeError > Error Value: manage_addZSQLMethod > >I studies the Znolk product and found that it uses _setObject directly >instead of >manage_addZSQLMethod . > In general there is an inconsistency in adding objects: sometimes you can use the manage_addYourProduct interface, and sometimes only going through the addProduct invocation (don't have the whole thing handy here) will do the trick. This can get very confusing. Wouldn't know if that's the matter here, though. I think this should be standardized in the interfaces. >but it IMHO the visibility of manage_addDTMLMethod and manage_addZSQLMethod >_should_ be the same ? > I agree

1 0

why is manage_addZSQLMethod unavailable ?
by Hannu Krosing 18 Aug '00

18 Aug '00

Why does this work <dtml-call "_['testadd'].manage_addDTMLMethod('MethodId', 'Method Title','method text')"> While this does not: <dtml-call "_['testadd'].manage_addZSQLMethod('ZSQLID', 'ZSQL Title', 'DB','','select * from data')"> Error Type: AttributeError Error Value: manage_addZSQLMethod I studies the Znolk product and found that it uses _setObject directly instead of manage_addZSQLMethod . I was able to overcome it by defining an one-line external method: ----------------- from Products.ZSQLMethods.SQL import * def manage_addZSQLMethod(self, sqlId, sqlTitle, connection_id, args, command): self._setObject(sqlId, SQL(sqlId, sqlTitle, connection_id, args, command)) ----------------- but it IMHO the visibility of manage_addDTMLMethod and manage_addZSQLMethod _should_ be the same ? ------------- Hannu

1 0

Caching problems
by Bob Pepin 18 Aug '00

18 Aug '00

Hi, I have a problem with a class I wrote where I have a list as an attribute of the class. When I append something to that list, it stays there at first, but only until I restart Zope. It disappears (==is set to the value I assigned to it in __init__) and reappears as well when I hit reload a few times very quickly. Whenever I flush the cache it disappears immediately. There seems to be no transaction registered by Zope, because it doesn't show up under 'Undo'. I observed this both thru a dtml page and a debugging function written in python. I attached the code below, the method and attributes I'm talking about are IEEShare.read_access_roles, IEEShare.write_access_roles and IEEShare.add_user_access() The problem exists with both Zope 2.2.0 and 2.2.1b1. I'm running 2.2.1b1 right now on a SuSE Linux 6.4 default installation. (standard libc, threads etc.) Both versions of Zope are compiled from source.

5 5

"stemmed and stopped": problems with stopwords and the 'and' operator
by R. David Murray 17 Aug '00

17 Aug '00

OK, so the TextIndex of a ZCatalog says that it "stems and stops" the words before indexing them and, one would hope, before searching for them. I always thought that "stem" meant "derive the stem of the word" (so as to make the index smaller). I just peeked at the Splitter.c source code for the first time, and that sure ain't it. The American phrase would be "truncate and stop", I think. In any case, "stem" in the source code comments means truncate at MAX_WORD, which is 64 characters. That's an aside. Now, about stopping. There is a list of "stop words" that don't get indexed. Fine. I'm having quite a bit of trouble figuring out exactly where this is happening, but let's ignore that for now on the indexing side. It happens, that's enough for now. Now, what happens to stop words in an input search string? >From my single stepping the code, stopwords are still in the query string while it is being parsed, and get looked up in the index. So, here is the heart of my problem: consider the search string someword and someotherword Suppose 'someword' is a stopword. It doesn't get indexed because it is considered too common. Now, I would think that if this search string is submitted, the result would be to return the hits for 'someotherword'. This might, however, not be other people's opinions. So, is the fact that TextIndex appears to return the null set in this case a bug or a feature? I say 'appears' because I actually get 2 (out of about 2000 with the keyword 'car') hits in my database when I search on 'car and the'. I tried to single step through the logic using the debugger, but when the call is made made to the splitter with the stopword passed in, python core dumps. I can do 'from SearchIndex.Splitter import Splitter', and call Splitter, and see that stopwords are not removed, but I can't do 'from SearchIndex.UnTextIndex import Splitter' because it complains about not being able to import Persistent from Persistence. (*That* problem was reported by someone else in another context not too long ago.) However, it's pretty clear that this null set return is what is happening, since when the evaluate subroutine is entered, the stop word is in the partially parsed string, and is in fact passed to the Splitter in the __getitem__ of the text index. If the splitter stopped it, the returned result set would be None, If the splitter doesn't stop it, the text index is still going return a null set as the result for that word, since it doesn't appear in the index by definition. An 'and' of any result set with None is going to be the null set. So it looks like the thing was designed this way: the stop words get "deleted" from the search string by not being in the index and by therefore returning null sets when looked up. This works fine for 'or' logic, but not for 'and' logic, IMO. Contrary opinions? Helpful hints? If I'm right and this needs fixed, it's going to be a bit of a bear to do, I think. (Where those two hits are coming from is a *real* mystery, but one I'm going to ignore for a little while yet since I can't yet get the debugger to work for me without crashing. I have a sneaking suspicion it is related to my confusion about where stopwords get removed in the indexing process, but it will probably take a while for me to prove or disprove that notion.) --RDM

2 2

Data.fs
by Andre Schubert 17 Aug '00

17 Aug '00

Hi, I'am using Zope 2.1.6 and my Data.fs is going corrupted and i don't have found a way for backup this file. My Questions are: is there a way to delete the last transaction from Data.fs, and can i backup the Data.fs when Zope is running. Is the backup of Data.fs inconsitent when i make the backup at the same time a user writes to it? I want to a backup Data.fs twice a day. so i want to have 14 data.fs files every week. On this way i can find out when the error occurs and i can restore an older version of Data.fs. as

2 1

patch for &dtml-foo:bar:baz;
by Steve Alexander 17 Aug '00

17 Aug '00

This patch changes lib/python/DocumentTemplate/DT_HTML.py so that you can use paths for traversal in DTML entity syntax. The delimiter is ':', as it is valid as part of an XML entity, but not valid as part of a Zope object id. You can use &dtml-:foo:bar; to ensure that traversal occurs from the root object. This patch is against Zope 2.2.1b1. It relies on the restrictedTraverse() api. A 2.1.x version would be possible, but would be rather messy. -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net *** DT_HTML.py.old --- DT_HTML.py *************** *** 98,104 **** end_match=regex.compile('[\0- ]*$/\|end$', regex.casefold).match, start_search=regex.compile('[<&]').search, ! ent_name=regex.compile('[-a-zA-Z0-9_.]+').match, find=find, strip=strip, replace=replace, --- 98,104 ---- end_match=regex.compile('[\0- ]*$/\|end$', regex.casefold).match, start_search=regex.compile('[<&]').search, ! ent_name=regex.compile('[-a-zA-Z0-9_.:]+').match, find=find, strip=strip, replace=replace, *************** *** 156,162 **** d[1]=d['end']='' d[2]=d['name']='var' d[0]=text[s:e+1] ! d[3]=d['args']=args+' html_quote' return s else: nn=find(args,'-') --- 156,165 ---- d[1]=d['end']='' d[2]=d['name']='var' d[0]=text[s:e+1] ! d[3]=d['args']=':' in args \ ! and '"_.render(restrictedTraverse(\''+ \ ! replace(args,':','/')+'\'))" html_quote' \ ! or args+' html_quote' return s else: nn=find(args,'-') *************** *** 164,171 **** d[1]=d['end']='' d[2]=d['name']='var' d[0]=text[s:e+1] ! args=(args[nn+1:]+' '+ ! replace(args[:nn],'.',' ')) d[3]=d['args']=args return s --- 167,178 ---- d[1]=d['end']='' d[2]=d['name']='var' d[0]=text[s:e+1] ! path=args[nn+1:] ! args=(':' in path ! and '"_.render(restrictedTraverse(\''+ ! replace(path,':','/')+'\'))" ' ! or path+' ') \ ! +replace(args[:nn],'.',' ') d[3]=d['args']=args return s

1 0