Re: [Zope-Annce] ANNOUNCE: Zope 2.3.1 beta 2 released
Hello! On Fri, 16 Mar 2001, Oleg Broytmann wrote:
http://www.zope.org/Products/Zope/2.3.1b2/ http://www.zope.org/Members/mcdonc/HowTos/UpgradeToNewCatalog
I have problems with new catalog. I successfully upgraded to beta2. I ran catalog/manage_convertBTrees. It worked, but my catalog lost all objects. I tried to Find Items. No items found. I completely removed the catalog and recreated another one with the same name. I found a bug - the Undo tab shows operation to undo. That's bad - I don't wnat to undo manage_convertBTrees on newly created catalog. Well, I ran Find Objects. 365 objects found. I performed few test searches. Almost all goes weel, with one bug. I can search "python" and "python*". I can search with russian letters (I repatched Splitter.c to be locale-aware). But I can't serach russian text with * <h2>Zope Error</h2> <p>Zope has encountered an error while publishing this resource.</p> <p> <strong>Error Type: TextIndex.QueryError</strong><br> <strong>Error Value: Malformed query</strong><br> </p> <hr noshade> <p>Troubleshooting Suggestions</p> <ul> <li>The URL may be incorrect.</li> <li>The parameters passed to this resource may be incorrect.</li> <li>A resource that this resource relies on may be encountering an error.</li> </ul> <p>For more detailed information about the error, please refer to the HTML source for this page. </p> <p>If the error persists please contact the site maintainer. Thank you for your patience. </p> </td></tr> </table> </BODY></HTML> <PRE> Traceback (innermost last): File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/Publish.py, line 187, in publish File /usr/local/Zope-2.3.1b2-src/lib/python/Zope/__init__.py, line 221, in zpublisher_exception_hook (Object: Traversable) File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/Publish.py, line 171, in publish File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: search-results) File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: search-results) File /usr/local/Zope-2.3.1b2-src/lib/python/OFS/DTMLMethod.py, line 189, in __call__ (Object: search-results) File /usr/local/Zope-2.3.1b2-src/lib/python/DocumentTemplate/DT_String.py, line 538, in __call__ (Object: search-results) File /usr/local/Zope-2.3.1b2-src/lib/python/DocumentTemplate/DT_Let.py, line 147, in render (Object: catalog="_.getitem('search-catalog', 0)") File /usr/local/Zope-2.3.1b2-src/lib/python/DocumentTemplate/DT_Util.py, line 334, in eval (Object: REQUEST.set('search1', catalog(PrincipiaSearchSource=text_search))) (Info: REQUEST) File <string>, line 0, in ? File /usr/local/Zope-2.3.1b2-src/lib/python/Products/ZCatalog/ZCatalog.py, line 530, in searchResults (Object: Traversable) File /usr/local/Zope-2.3.1b2-src/lib/python/Products/ZCatalog/Catalog.py, line 654, in searchResults File /usr/local/Zope-2.3.1b2-src/lib/python/Products/ZCatalog/Catalog.py, line 545, in _indexedSearch File /usr/local/Zope-2.3.1b2-src/lib/python/SearchIndex/UnTextIndex.py, line 492, in _apply_index File /usr/local/Zope-2.3.1b2-src/lib/python/SearchIndex/UnTextIndex.py, line 546, in query File /usr/local/Zope-2.3.1b2-src/lib/python/SearchIndex/UnTextIndex.py, line 624, in evaluate TextIndex.QueryError: (see above) Oleg. ---- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
----- Original Message ----- From: "Oleg Broytmann" <phd@phd.russ.ru> To: <phd@mail2.phd.pp.ru> Cc: "Zope Mailing List" <zope@zope.org> Sent: Friday, March 16, 2001 10:36 AM Subject: [Zope] Re: [Zope-Annce] ANNOUNCE: Zope 2.3.1 beta 2 released
Hello!
On Fri, 16 Mar 2001, Oleg Broytmann wrote:
http://www.zope.org/Products/Zope/2.3.1b2/ http://www.zope.org/Members/mcdonc/HowTos/UpgradeToNewCatalog
I have problems with new catalog. I successfully upgraded to beta2. I ran catalog/manage_convertBTrees. It worked, but my catalog lost all objects. I tried to Find Items. No items found.
Clearly this is a bad thing. Sorry. I did test this and it worked for me. Looks like we'll need to take another look at the conversion stuff.
I completely removed the catalog and recreated another one with the same name. I found a bug - the Undo tab shows operation to undo. That's bad - I don't wnat to undo manage_convertBTrees on newly created catalog.
Actually, maybe you do. ;-) In the case that you weren't able to live with the fact that all of your items were munched by the conversion routine, this would be handy. I think this transaction had to do with the call to manage_convertBTrees called prior to the new catalog add.
Well, I ran Find Objects. 365 objects found. I performed few test searches. Almost all goes weel, with one bug. I can search "python" and "python*". I can search with russian letters (I repatched Splitter.c to be locale-aware). But I can't serach russian text with *
Hmmm. Can you provide a pointer to the patched splitter.c source? How do we test this?
<h2>Zope Error</h2> <p>Zope has encountered an error while publishing this resource.</p>
<p> <strong>Error Type: TextIndex.QueryError</strong><br> <strong>Error Value: Malformed query</strong><br> </p>
<hr noshade>
<p>Troubleshooting Suggestions</p>
<ul> <li>The URL may be incorrect.</li> <li>The parameters passed to this resource may be incorrect.</li> <li>A resource that this resource relies on may be encountering an error.</li> </ul>
<p>For more detailed information about the error, please refer to the HTML source for this page. </p>
<p>If the error persists please contact the site maintainer. Thank you for your patience. </p> </td></tr> </table>
</BODY></HTML> <PRE> Traceback (innermost last): File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/Publish.py, line 187, in publish File /usr/local/Zope-2.3.1b2-src/lib/python/Zope/__init__.py, line 221, in zpublisher_exception_hook (Object: Traversable) File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/Publish.py, line 171, in publish File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: search-results) File /usr/local/Zope-2.3.1b2-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: search-results) File /usr/local/Zope-2.3.1b2-src/lib/python/OFS/DTMLMethod.py, line 189, in __call__ (Object: search-results) File /usr/local/Zope-2.3.1b2-src/lib/python/DocumentTemplate/DT_String.py, line 538, in __call__ (Object: search-results) File /usr/local/Zope-2.3.1b2-src/lib/python/DocumentTemplate/DT_Let.py, line 147, in render (Object: catalog="_.getitem('search-catalog', 0)") File /usr/local/Zope-2.3.1b2-src/lib/python/DocumentTemplate/DT_Util.py, line 334, in eval (Object: REQUEST.set('search1', catalog(PrincipiaSearchSource=text_search))) (Info: REQUEST) File <string>, line 0, in ? File /usr/local/Zope-2.3.1b2-src/lib/python/Products/ZCatalog/ZCatalog.py, line 530, in searchResults (Object: Traversable) File
/usr/local/Zope-2.3.1b2-src/lib/python/Products/ZCatalog/Catalog.py,
line 654, in searchResults File /usr/local/Zope-2.3.1b2-src/lib/python/Products/ZCatalog/Catalog.py, line 545, in _indexedSearch File /usr/local/Zope-2.3.1b2-src/lib/python/SearchIndex/UnTextIndex.py, line 492, in _apply_index File /usr/local/Zope-2.3.1b2-src/lib/python/SearchIndex/UnTextIndex.py, line 546, in query File /usr/local/Zope-2.3.1b2-src/lib/python/SearchIndex/UnTextIndex.py, line 624, in evaluate TextIndex.QueryError: (see above)
Oleg. ---- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Fri, 16 Mar 2001, Chris McDonough wrote:
Well, I ran Find Objects. 365 objects found. I performed few test searches. Almost all goes weel, with one bug. I can search "python" and "python*". I can search with russian letters (I repatched Splitter.c to be locale-aware). But I can't serach russian text with *
Hmmm. Can you provide a pointer to the patched splitter.c source? How do we test this?
I put the patch to the Collector and someone from DC (I think it was Brian Lloyd) promised to put it into CVS in next version. Well, it's not in 2.3.1b2. I cannot find it in the Collector - I got "Principia Unavailable" on the page http://classic.zope.org:8080/Collector/bugsAndFeatures/Classification/Bug/ta... (did you upgraded to beta2? :) and I cannot search, because Collector asks issue id. I don't know id. The patch is perfectly simple and cannot destroy searching, I beleive. This is for Splitter.c; I am providing here simple patch, not a context diff to make it simpler for eyes; in the Collector it is in diff -c format: 228c228 < while (isspace(*here) && (here < end)) here++; ---
while (isspace((unsigned char)*here) && (here < end)) here++;
232c232 < c=tolower(*here); ---
c=tolower((unsigned char)*here);
Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
I put the patch to the Collector and someone from DC (I think it was Brian Lloyd) promised to put it into CVS in next version. Well, it's not in 2.3.1b2.
I never promise anyone anything for a specific release, unless it is already done :)
I cannot find it in the Collector - I got "Principia Unavailable" on the page http://classic.zope.org:8080/Collector/bugsAndFeatures/Classificat ion/Bug/tableBrowse (did you upgraded to beta2? :) and I cannot search, because Collector asks issue id. I don't know id.
No, I added direct search by issue id to the Collector yesterday and made a boo-boo. It's fixed now. The issue id is 2022, and it looks like it was marked accepted by Chris on Mar 5. I've gone ahead and merged the patch and marked it completed. Brian Lloyd brian@digicool.com Software Engineer 540.371.6909 Digital Creations http://www.digicool.com
Egad, I remember this patch. It should have gone in. Amidst all the other stuff, I neglected to apply it. I will go do so now. This still doesn't help us figure out why the searching of russian characters, however, didn't work even after you applied the patch, correct? Maybe you can provide a patch to the unit testing stuff that demonstrates the failure? Or a simple demonstration that we can turn into a unit test?
The patch is perfectly simple and cannot destroy searching, I beleive. This is for Splitter.c; I am providing here simple patch, not a context diff to make it simpler for eyes; in the Collector it is in diff -c format:
228c228 < while (isspace(*here) && (here < end)) here++; ---
while (isspace((unsigned char)*here) && (here < end)) here++; 232c232 < c=tolower(*here);
c=tolower((unsigned char)*here);
On Fri, 16 Mar 2001, Chris McDonough wrote:
This still doesn't help us figure out why the searching of russian characters, however, didn't work even after you applied the patch, correct?
I applied the patch and the upgraded Catalog. BTW, I forget to mention - I run Zope on Python 2.0. Can it cause this problem?
Maybe you can provide a patch to the unit testing stuff that demonstrates the failure? Or a simple demonstration that we can turn into a unit test?
Where are unit tests? Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
On Fri, 16 Mar 2001, Chris McDonough wrote:
This still doesn't help us figure out why the searching of russian characters, however, didn't work even after you applied the patch, correct?
I applied the patch and the upgraded Catalog. BTW, I forget to mention - I run Zope on Python 2.0. Can it cause this problem?
I'm not sure. Might it have something to do with Unicode stuff? I have no experience whatsoever with alternate charactersets. Do you think you can try it under 1.5.2?
Maybe you can provide a patch to the unit testing stuff that
demonstrates
the failure? Or a simple demonstration that we can turn into a unit test?
Where are unit tests?
In the "lib/python/SearchIndex/tests directory and the "lib/python/Products/ZCatalog/tests" directory of 2.3.1b2+.
On Fri, 16 Mar 2001, Chris McDonough wrote:
I run Zope on Python 2.0. Can it cause this problem?
I'm not sure. Might it have something to do with Unicode stuff? I have no experience whatsoever with alternate charactersets. Do you think you can try it under 1.5.2?
I'll try it next week.
Where are unit tests?
In the "lib/python/SearchIndex/tests directory and the "lib/python/Products/ZCatalog/tests" directory of 2.3.1b2+.
Will look into into it, but no early than Monday. It is 20:45 here, and I'll be off-line this weekend. Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Where are unit tests?
In the "lib/python/SearchIndex/tests directory and the "lib/python/Products/ZCatalog/tests" directory of 2.3.1b2+.
Will look into into it, but no early than Monday. It is 20:45 here, and I'll be off-line this weekend.
The unit tests are pretty complicated. If you can't figure them out or where to put a test, a command-line demonstration of the problem at the Python prompt would be just as good (we'll turn it into a unit test).
On Fri, 16 Mar 2001, Chris McDonough wrote:
The unit tests are pretty complicated. If you can't figure them out or where to put a test, a command-line demonstration of the problem at the Python prompt would be just as good (we'll turn it into a unit test).
How can I test Catalog from command-line? Create a minimal ZODB and index it? Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Load a ZODB with stuff... the cd to lib/python invoke python
import Zope app = Zope.app() (and pretending your catalog is named "acatalog" in the Zope root) app.acatalog.searchResults({'someindexname':'some text', 'someotherindexname':'some other text'})
If you can boil it down to the simplest query that results in the error (and let us know what kinds of indexes your indexnames respectively refer to), we can build a unit test that exhibits the problem and fix it. ... that ----- Original Message ----- From: "Oleg Broytmann" <phd@phd.pp.ru> To: "Chris McDonough" <chrism@digicool.com> Cc: "Zope Mailing List" <zope@zope.org> Sent: Friday, March 16, 2001 12:58 PM Subject: Re: [Zope] Re: [Zope-Annce] ANNOUNCE: Zope 2.3.1 beta 2 released
On Fri, 16 Mar 2001, Chris McDonough wrote:
The unit tests are pretty complicated. If you can't figure them out or where to put a test, a command-line demonstration of the problem at the Python prompt would be just as good (we'll turn it into a unit test).
How can I test Catalog from command-line? Create a minimal ZODB and index it?
Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Hi! On Fri, 16 Mar 2001, Chris McDonough wrote:
The unit tests are pretty complicated. If you can't figure them out or where to put a test, a command-line demonstration of the problem at the Python prompt would be just as good (we'll turn it into a unit test).
I started with triing to run the test as is from command-line. I created zope.mbox full of my old russian emails and ran the test. It failed: checkTimeBulkIndex (__main__.TestTimeIndex) ... loading data took 0.185411930084 seconds.. bulk index took 0.881116986275 seconds.. total time for load and index was 1.06652891636 seconds.. ok checkTimeIncrementalIndexAndQuery (__main__.TestTimeIndex) ... total time for load and index was 0.973206996918 seconds.. ====================================================================== FAIL ---------------------------------------------------------------------- Traceback (innermost last): File "testCatalogTiming.py", line 95, in checkTimeIncrementalIndexAndQuery assert len(rs) == max, len(rs) AssertionError: 0 ====================================================================== checkTimeSubcommit (__main__.TestTimeIndex) ... testing subcommit at theshhold of None total time with subcommit thresh None was 0.161839962006 seconds.. testing subcommit at theshhold of 100 total time with subcommit thresh 100 was 0.182608962059 seconds.. testing subcommit at theshhold of 500 total time with subcommit thresh 500 was 0.163025021553 seconds.. testing subcommit at theshhold of 1000 total time with subcommit thresh 1000 was 0.229267001152 seconds.. testing subcommit at theshhold of 10000 total time with subcommit thresh 10000 was 0.162956953049 seconds.. ok ------------------------------------------------------------------------------ Ran 3 tests in 51.696s FAILED (failures=1) Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Hi! On Fri, 16 Mar 2001, Chris McDonough wrote:
I applied the patch and the upgraded Catalog. BTW, I forget to mention - I run Zope on Python 2.0. Can it cause this problem?
I'm not sure. Might it have something to do with Unicode stuff? I have no experience whatsoever with alternate charactersets. Do you think you can try it under 1.5.2?
rm -rf / Clean installation, untar, python1.5 w_pcgi.py, patch Splitter.c, make, copy Data.fs, start -L koi8-r (russian locale). catalog/manage_convertBTrees - no items in catalog :( Find items - no items found :((( Remove the catalog, create a new one Find items - 365 items found. Search works - including search russian texts with *. Resume. 1. manage_convertBTrees is broken, regardless of python 1.5.2 or 2.0. 2. Search is somewhat broken with python 2.0 - I cannot search russian texts with *. All other searches work - english text, english text with *, and russian text without *. PS. I'll return to catalog unit test later. Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg, do you *really* want to do 'rm -rf /', seems a bit extreme to me ;) Phil ----- Original Message ----- From: "Oleg Broytmann" <phd@phd.pp.ru> To: "Chris McDonough" <chrism@digicool.com> Cc: "Zope Mailing List" <zope@zope.org> Sent: Monday, March 19, 2001 11:03 AM Subject: Re: [Zope] Re: [Zope-Annce] ANNOUNCE: Zope 2.3.1 beta 2 released
Hi!
On Fri, 16 Mar 2001, Chris McDonough wrote:
I applied the patch and the upgraded Catalog. BTW, I forget to mention - I run Zope on Python 2.0. Can it cause this problem?
I'm not sure. Might it have something to do with Unicode stuff? I have no experience whatsoever with alternate charactersets. Do you think you can try it under 1.5.2?
rm -rf / Clean installation, untar, python1.5 w_pcgi.py, patch Splitter.c, make, copy Data.fs, start -L koi8-r (russian locale).
catalog/manage_convertBTrees - no items in catalog :( Find items - no items found :(((
Remove the catalog, create a new one Find items - 365 items found. Search works - including search russian texts with *.
Resume. 1. manage_convertBTrees is broken, regardless of python 1.5.2 or 2.0.
2. Search is somewhat broken with python 2.0 - I cannot search russian texts with *. All other searches work - english text, english text with *, and russian text without *.
PS. I'll return to catalog unit test later.
Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Other than this joke, all other was real; manage_convertBTrees is broken for me :( On Mon, 19 Mar 2001, Phil Harris wrote:
do you *really* want to do 'rm -rf /', seems a bit extreme to me ;)
rm -rf / Clean installation, untar, python1.5 w_pcgi.py, patch Splitter.c, make, copy Data.fs, start -L koi8-r (russian locale).
catalog/manage_convertBTrees - no items in catalog :( Find items - no items found :(((
Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
participants (5)
-
Brian Lloyd -
Chris McDonough -
Oleg Broytmann -
Oleg Broytmann -
Phil Harris