- Zope - Zope lists

BTreeFolders + Catalog + lots of objects?
by sean.upton＠uniontrib.com 08 Nov '01

08 Nov '01

Has anyone used the BTreeFolder product to store hundreds-of-thousands or millions of objects? I'm developing an internal CRM system that will contain somewhere between 300k-500k records, stored in a back-end relational datastore, and exposed via metadata proxy objects (1-per-record) sitting in a container subclassed from BTreeFolder; these objects will have meta-data cataloged in a ZCatalog for a data management interface. All this will sit in a non-undo storage (for now, Ty Sarna's Berkeley DB 2 storage, since that is what I am using now on my production ZEO storage servers, but eventually it will be migrated to a bsddb3 packless storage). These metadata proxy objects would likely be fully recreated from the underlying relational database on a weekly basis. I am wondering if anybody has done anything similar to this, in terms of number of objects stored in a BTreeFolder, and the type of storage that they used. I'm also wondering about anyone using ZCatalog for such a large number of indexed objects. I'm also thinking of overriding BTreeFolder.manage_main_listing() with a user interface that allows users to simply type in an object id into a text box, instead of listing them (the string object ids correspond to a long integer from 1..n); it wouldn't work very well to list half-a-million objects in a html-form select control... Ideally, I'd do batching of some sort, but I'm having trouble figuring out how I would do that. Anyone have any thoughts? Sean ========================= Sean Upton Senior Programmer/Analyst SignOnSanDiego.com The San Diego Union-Tribune 619.718.5241 sean.upton(a)uniontrib.com =========================

2 1

Re: [Zope] Re: Threading, dtml and performance?!
by Oliver Erlewein 08 Nov '01

08 Nov '01

Hi Tony No, I havn't solved the problem but after restarting production (after 19 days) I noticed that the funny way the threadcount reacted. I'd say, that the threads don't get killed after they have finished. I also noticed that the memory use went down from 344Mb to about 26Mb. Now I've put a restart into my nightly cronjob. (Are there some reasons why Zope should not be restarted dayly?????) I've just tested the whole thing on a WinNT 4.0 PC with Zope 2.4.2 (binary distribution). No problems! The PC gets slower but can answer two browser instances parallel. If I do the same thing on Solaris (intel & sun servers!) it is impossible! Normally you have sortof a break in a loop where control is given to the other parallel threads. This seems not to happen on my source compile with Solaris. I'm not so firm on where to address this problem (or if the problem was already adressed elswhere e.g. python maillists). Therefore I'd really apreciate some help! ;-) I don't want to start thinking about a migration Linux or even (puke) Windows. On Solaris I'm still using 2.4.0. Does it make sense to upgrade? I've seen that in Zope 2.4.3b1 there was some issue with dtml-recursion? Greetings from a rany Cologne! Oliver -- GMX - Die Kommunikationsplattform im Internet. http://www.gmx.net

3 3

Re: [Zope] Zope Server Crashes
by Anthony Baxter 08 Nov '01

08 Nov '01

Ok, couple of things: Do you get core files when Zope crashes? If so, can you get a gdb backtrace? There's at least one known crash-causing bug in Python2.1.1 that will be fixed in Python2.1.2 - it's pretty obscure, but it can happen. Without a gdb backtrace, it's very very hard to say what the problem might be. Make sure that any external packages you're using (database adapters, in particular) have been upgraded to python2.1. Anthony >>> "Jean Pierre Litsas" wrote > I am having the same problem and i am wasting my time trying running zope > since october 29 and nothing, zope crashes every time, i did all the things > that this list advice and zope do not run more than 10 minutes, > i think that version 2.4.2 is so BAD, i compiled python2 > with --without-pymalloc and installed it, i compiled zope and installed, i > compiled PgSQL and installed, i compiled mxDateTime and other things more > but nothing works,i want the old and good zope back.... 2.4.2 is SO BAD. I > dont know what to do, i think i will install 2.4.0 back again. > > My system: > RedHat 7.2 > Postgres 7.1.3 > PgSQL 1.6 > Python 2.1.1 > > > > > > ----- Original Message ----- > From: <sean.upton(a)uniontrib.com> > To: <Michelle.Jacobs(a)ftech.net>; <zope(a)zope.org> > Sent: Monday, November 05, 2001 2:01 PM > Subject: RE: [Zope] Zope Server Crashes > > > > 2 quick suggestions for some detective work: > > > > - watch your memory profile for Zope; determine if this is a problem (i.e. > a > > leak in an app or Catalog, etc, use a tool like top) > > - enable "big M" logging... you can search the list archives for more > > info... > > > > Sean > > > > -----Original Message----- > > From: Michelle Jacobs [mailto:Michelle.Jacobs@ftech.net] > > Sent: Monday, November 05, 2001 8:01 AM > > To: zope(a)zope.org > > Subject: [Zope] Zope Server Crashes > > > > > > > > Hi, > > > > We've been running Zope on Linux since March 2001 and we've been quite > > pleased - even put our company web site on Zope. However, over the past > > few weeks we've experienced more and more frequent server crashes until > > finally, out of desperation, we upgraded from 2.3.2 to 2.4.2. The > > upgrade seemed to solve the problem temporarily, but we've been > > experiencing more and more crashes over the last few days. We've now > > installed a script which checks for failed requests and restarts Zope in > > the event of a crash, but the problem is only becoming worse. Server > > logs show only gaps and not much in the way of useful information while > > the server was down. > > > > Anyone out there have experience of this sort of thing with Zope or can > > offer any suggestions? > > > > > > Very Gratefully Yours, > > Michelle Jacobs > > > > -- > > Michelle Jacobs - Development Manager - Frontier Internet Services > > Limited > > Tel: 029 20 820 005 Fax: 029 20 820 037 http://www.frontier.net.uk > > Statements made are at all times subject to Frontier's Terms and > > Conditions of Business, which are available upon request. > > > > > > > > > > _______________________________________________ > > Zope maillist - Zope(a)zope.org > > http://lists.zope.org/mailman/listinfo/zope > > ** No cross posts or HTML encoding! ** > > (Related lists - > > http://lists.zope.org/mailman/listinfo/zope-announce > > http://lists.zope.org/mailman/listinfo/zope-dev ) > > > > _______________________________________________ > > Zope maillist - Zope(a)zope.org > > http://lists.zope.org/mailman/listinfo/zope > > ** No cross posts or HTML encoding! ** > > (Related lists - > > http://lists.zope.org/mailman/listinfo/zope-announce > > http://lists.zope.org/mailman/listinfo/zope-dev ) > > > > > _______________________________________________ > Zope maillist - Zope(a)zope.org > http://lists.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope-dev ) > -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

1 0

Re: [Zope] WebDAV lockingproblem with Word
by Carl Robert Blesius 08 Nov '01

08 Nov '01

I am so happy to see that there is some momentum building up behind a problem that prevents wider spread Zope use. Hip, Hip Hurray! I wish I knew how to do more than just cheer... but at least it is something. :-) > I have been looking trough the Microsoft developer site and found a > reference that might be helpful > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wss/wss/_w… > > I tried to compile one of their C++ examples but failed since 2 > identifiers are undeclared > IID_IXMLHttpRequest > and > CLSID_XMLHTTPRequest > I found their declaration nowhere. > So if somebody could tell me what I need to have them declared I could > start fiddling. > > Robert

1 0

install zope2.4.2 on Redhat error (again)
by Tom Cameron 08 Nov '01

08 Nov '01

I sent this message off to the list a short while ago and got no response. I have since been emailed by someone else who has experienced the same problem. Can anybody out there help us solve this one? TIA Tom Cameron => -----Original Message----- => From: zope-admin(a)zope.org [mailto:zope-admin@zope.org]On Behalf Of Tom => Cameron => Sent: Thursday, 1 November 2001 4:23 PM => To: zope(a)zope.org => Subject: [Zope] install zope2.4.2 on Redhat error => => => I just downloaded the zope binary and ran the install script and => I got the => following error. => => What does this mean?? => => Zope appears to be running fine with no problems, but I am certainly => concerned that this occured on the install. => => Tom => => [Zope-2-4-2]# ./install => => ----------------------------------------------------------------- => ----------- => -- => Compiling python modules => File "/home/zope/Zope-2-4-2/lib/python2.1/test/nocaret.py", line 2 => [x for x in x] = x => SyntaxError: can't assign to list comprehension => SyntaxError: from __future__ imports must occur at the beginning => of the file => (test_future3.py, line 3) => SyntaxError: from __future__ imports must occur at the beginning => of the file => (test_future4.py, line 3) => SyntaxError: from __future__ imports must occur at the beginning => of the file => (test_future5.py, line 4) => SyntaxError: from __future__ imports must occur at the beginning => of the file => (test_future6.py, line 3) => SyntaxError: from __future__ imports must occur at the beginning => of the file => (test_future7.py, line 3) => ----------------------------------------------------------------- => ----------- => -- => ----------------------------------------------------------------- => ----------- => -- => creating default inituser file => Note: => The initial user name and password are 'admin' => and 'XXXXXXXXX'. => => You can change the name and password through the web => interface or using the 'zpasswd.py' script. => => chmod 0600 /home/zope/Zope-2-4-2/inituser => chmod 0711 /home/zope/Zope-2-4-2/var => ----------------------------------------------------------------- => ----------- => -- => setting dir permissions => ----------------------------------------------------------------- => ----------- => -- => creating default database => chmod 0600 /home/zope/Zope-2-4-2/var/Data.fs => ----------------------------------------------------------------- => ----------- => -- => Writing the pcgi resource file (ie cgi script), => /home/zope/Zope-2-4-2/Zope.cgi => chmod 0755 /home/zope/Zope-2-4-2/Zope.cgi => ----------------------------------------------------------------- => ----------- => -- => Creating start script, start => chmod 0711 /home/zope/Zope-2-4-2/start => ----------------------------------------------------------------- => ----------- => -- => Creating stop script, stop => chmod 0711 /home/zope/Zope-2-4-2/stop => ----------------------------------------------------------------- => ----------- => -- => => => => _______________________________________________ => Zope maillist - Zope(a)zope.org => http://lists.zope.org/mailman/listinfo/zope => ** No cross posts or HTML encoding! ** => (Related lists - => http://lists.zope.org/mailman/listinfo/zope-announce => http://lists.zope.org/mailman/listinfo/zope-dev ) =>

2 1

Shopping Cart
by Ben Ocean 07 Nov '01

07 Nov '01

Hi; I need a shopping cart for an app I'm building in Zope. I've worked with a cart built in PHP and been quite happy with it. I notice that the only product currently available for this app in Zope is zCommerce. There has only been one developmental release of it and that was well over a year ago. Has anyone had any good experience with this app? Or, would I be better off trying to port the PHP app into Zope? Or proxy-passing back and forth between Zope and Apache? TIA, BenO

3 3

Re: [Zope] Zope hiccuping
by Chris Kratz 07 Nov '01

07 Nov '01

OK, did a ./configure --without-py_malloc with python 2.1.1 and it didn't take care of the problem. Still getting: 2001-11-07T20:30:11 ERROR(200) zdaemon zdaemon: Wed Nov 7 15:30:11 2001: Aiieee! 9400 exited with error code: 11 Yes, it did compile (make clean, configure, make) and the Makefile has the argument --without-py_malloc The start script is pointed to the newly compiled python and I'm still getting the same error. Aack! -chris ------------------------------ Chris Kratz chris.kratz(a)vistashare.com ----- Original Message ----- From: "Chris Kratz" <chris.kratz(a)vistashare.com> To: "Chris Kratz" <chris.kratz(a)vistashare.com> Sent: Wednesday, November 07, 2001 2:49 PM Subject: Re: [Zope] Zope hiccuping > After posting this, I ran into some messages posted on the mailing list that > seem to say that py_malloc is the culprit for this particular problem. I'm > in the process of trying this solution. My apologies for the spam. On the > other hand if there are any other thoughts, I would appreciate hearing them > as well. > > -Chris > ------------------------------ > Chris Kratz > chris.kratz(a)vistashare.com > > > ----- Original Message ----- > From: "Chris Kratz" <chris.kratz(a)vistashare.com> > To: <zope(a)zope.org> > Sent: Wednesday, November 07, 2001 2:33 PM > Subject: [Zope] Zope hiccuping > > > > We have been noticing that periodically, we get an error from IE that says > > "Cannot find server or DNS error". It is not easily reproducable (except > by > > just clicking links on the server) and a F5 refresh in the browser > [almost] > > always loads the page correctly. I turned on logging today with the -M > > startup option and observed the following entries when it happened: > > > > B 145697132 2001-11-07T17:15:38 GET /OutcomeTracker/Dev_News > > I 145697132 2001-11-07T17:15:38 0 > > A 145697132 2001-11-07T17:15:39 200 32155 > > E 145697132 2001-11-07T17:15:39 > > B 145934020 2001-11-07T17:15:40 GET > > /OutcomeTracker/PeopleOrganizations/index_html > > I 145934020 2001-11-07T17:15:40 0 > > B 135053764 2001-11-07T17:15:56 POST /OutcomeTracker/Activities/index_html > > I 135053764 2001-11-07T17:15:56 2831 > > B 146464388 2001-11-07T17:15:57 GET > > /OutcomeTracker/PeopleOrganizations/index_html > > I 146464388 2001-11-07T17:15:57 0 > > A 146464388 2001-11-07T17:16:03 200 31200 > > E 146464388 2001-11-07T17:16:03 > > > > Notice how the Get /OutcomeTracker/PeopleOrganizations/index_html never > gets > > the A or E lines, but only has a B and I line. The subsequent refresh > > finished the request. Interestingly, the two incompleted requests are not > > logged to z2.log. We can see the request before and the request after, > but > > that's it. The other strangeness is that in the postgres log, we see a > > "pq_recvbuf: unexpected EOF on client connection". This seemed to point > to > > zope threads dying. Since I'm not getting anything in the logs(*see > below), > > I started running tests with one eye on the currently running processes. > > And sure enough, whenever I got that error at the browser (cannot find > > server...), *All* of the zope threads (except the main starter thread) die > > quietly and come back with new PIDs. It really appears like it reruns the > > entire startup sequence again. With Z_DEBUG_MODE on I can watch it go > > through the startup sequence again whenever this happens. But, there are > no > > tracebacks. It's just like somebody clicked restart in the middle of a > > process. > > > > The one glimmer of hope is in the stupid log file: > > > > 2001-11-07T19:30:23 ERROR(200) zdaemon zdaemon: Wed Nov 7 14:30:23 2001: > > Aiieee! 1925 exited with error code: 11 > > ...restarting... > > > > Here's the questions, > > > > 1. It appears that something is causing those threads to crash (or end), > but > > nothing is getting put in the log file. Is there any way to get the > > tracebacks I assume are happening or to find out what is going on? > > 2. Alternatively, is there a way to run zope in single threaded mode? > > Z_DEBUG_MODE appears to only apply to the main thread because it goes > ahead > > and spawns additional threads. If I use -t 0 I get two processes, but no > > response from a web browser request. If I use -t 1, I get three processes > > owned by nobody and the original one by root. > > 3. Any further ideas on how to debug this thing? Where do I find what > error > > code 11 is? > > > > Thanks for you time and help, > > > > -Chris > > > > ------------------------------ > > Chris Kratz > > chris.kratz(a)vistashare.com > > > > > > > > _______________________________________________ > > Zope maillist - Zope(a)zope.org > > http://lists.zope.org/mailman/listinfo/zope > > ** No cross posts or HTML encoding! ** > > (Related lists - > > http://lists.zope.org/mailman/listinfo/zope-announce > > http://lists.zope.org/mailman/listinfo/zope-dev ) > > > >

4 5

Broken ZopeLabs?
by Chris Withers 07 Nov '01

07 Nov '01

"M. Adam Kendall" wrote: > > http://www.zopelabs.com/cookbook/996249320 Looks like zopelabs haven't closed off a table cell or something since the body of that page doesn't render in NS4 ;-) Chris

3 4

Create ZClass instance from python
by HNBeck＠t-online.de 07 Nov '01

07 Nov '01

Hi, I've created a product ("myItem"), containing a zclass "ItemType" with base class objectmanager. The python code is like this (got from some internet sources): context.manage_addProduct('myItem').ItemType_add(_.None,_, Redir=1) By executing the code there comes the error: "ItemType not defined". ???????? Anyone may help me ? Thanks !!! Grettings Hans ----------------------------------- Dipl.-Ing. Hans N. Beck Waldstr. 28 75045 Walzbachtal Tel: 07203/922280 Mobil: 0177/5383233 -----------------------------------

2 1

Re: [Zope] Pulling excel charts into zope/HTML
by Tim Hicks 07 Nov '01

07 Nov '01

> From: "Meilicke, Scott" <scott.meilicke(a)intp.com> > To: "'zope(a)zope.org'" <zope(a)zope.org> > Date: Wed, 7 Nov 2001 10:53:32 -0800 > Subject: [Zope] Pulling excel charts into zope/HTML > > Is anyone pulling excel charts into zope? I'm not pulling charts in as such, rather using the 'SaveAsHTML' function of Excel to spit out an entire worksheet (or whatever they're called) as html. > I'm *trying* to use com and zope on NT to export a chart into a gif and drop > it onto the file system with an external method. From there I can use > LocalFS or figure out a way to get it into the zodb for display. I've got things setup so that MS Office is used as a com object from a zope external method. This method creates a temporary directory into which it asks office to deposit the html and any images. The contents of this directory are then returned, at which point I use some dtml to do whatever I want (which is add the files to the ZODB in this case). Don't know if I've given you any great insight, but it works for me so... rgds tim

1 0