- Zope - Zope lists

Making transactions un-undoable ?
by Thierry FLORAC 31 Mar '03

31 Mar '03

Hi, I have a Zope site which interacts with a relational database (PostgreSQL). My obvious problem is that when some transactions are undone, concerned Zope objects are not synchronized anymore with the database, as SQL commands can't be undone. My question is : is there any way to make a specific transaction not undoable (without using a specific storage which may prohibit undo) ?? Thanks, Thierry -- Linux every day, keeps Dr Watson away... http://gpc.sourceforge.net -- http://www.ulthar.net

2 1

ZCTextIndex Lexicon
by volker.wend＠efgbsh.de 31 Mar '03

31 Mar '03

He folks, I somehow have a screwed up Zope Installation... I cannot create a ZCTextIndex Lexicon in a subfolder but I can create the lexikon in other folders on the same level. What did I do ? Thanks, Volker

3 2

Zope performance documentation
by Fabio Rolando 31 Mar '03

31 Mar '03

Hi. Does anybody know where I can found information about the performance of Zope as a web server? I mean, information like: number of documents that Zope can store and serve, max. access rate, max. users rate, hardware requirements, etc. Thank you in advance.

7 12

CorruptedDataError
by Jorg E. Rødsjø 31 Mar '03

31 Mar '03

I am running a number of squishdot sites for a university class, and I have problems. The this is that someone has probably done something wrong in their blog, and when I try to access the squishdot site I get the following error: [yes, I know zope is installed in a less than perfect directory, but the story is long] 2003-03-31T16:11:51 ERROR(200) ZODB Couldn't load state for '\x00\x00\x00\x00\x00\x00\xd9\xec' Traceback (innermost last): File /home/zope/zope261/Zope-2.6.1-linux2-x86/lib/python/ZODB/Connection.py, line 509, in setstate File /home/zope/zope261/Zope-2.6.1-linux2-x86/lib/python/ZODB/FileStorage.py, line 650, in load (Object: /home/zope/zope261/Zope-2.6.1-linux2-x86/var/Data.fs) File /home/zope/zope261/Zope-2.6.1-linux2-x86/lib/python/ZODB/FileStorage.py, line 630, in _load (Object: /home/zope/zope261/Zope-2.6.1-linux2-x86/var/Data.fs) CorruptedDataError: I have been in touch with squishdot support, but they said it was not likely something squshdot related. Does anyone have any idea what I can do? I cannot access the folder containing the squshdots either - the same error pops up. regards Jorg Rødsjø

2 1

ZCatalog performance issues - catalogging objects takes ages
by Wankyu Choi 31 Mar '03

31 Mar '03

Dear All, May I have your expertise on this? ;-) As much as I'm new to Zope/Python, ZCatalog (Catalog) internals vex me even more. I have a message board product called NeoBoard, some of you might know. Recently I rewrote its core to have a built-in catalog for indexing articles and displaying them automatically sorted on thread keys. It showed quite a boost in performance. Previous versions without the built-in catalog used to ramrod all article objects into/out of memory whenever they need to display them. What a waste of memory and CPU power as Toby Dickenson suggested. Here's what I did to solve this problem: - Rewrote the parent class of the NeoBoard/NeoBoardArticle ( article container/article objects themselves ), NeoPortalElementContainer to inherit ZCatalog. Basically NeoPortalElementContainer automatically natural sorts/numbers objects (elements) when they're added to the container: page_1, page_2, ... etc. - NeoBoardArticle looks toward NeoBoard when the catalog methods defined in NeoPortalElementContaier are called. So NeoBoard's catalog methods are always used no matter whereever you are in the path hierarchy. - When you call a NeoBoard instance, it calls ZCatalog's searchResults(), which returns brains objects. A threaded (expanded) look does require a step further: NeoBoard sorts a pageful of threads and their replies before returning them; it doesn't care about the other threads that are not displayed in the current request. Performance? Not so fast as SQL-backed PHP version ( displaying a pageful of threads takes only a fraction of a second ), but not bad. Okay, I partially solved one problem ( wasting memory/horsepower, etc - I'm still not satisfied with the performance, though ) but created another set of problems while so doing. I could display 5,000 threads ( about 20,000 article obects incuding all replies to the threads) in less than a second ( it takes a bit more when you load the board for the first time. ) The problems are... - It takes ages when cataloging even a small number of articles. 18 seconds for cataloging 50 or so article objects with so little to index? Is it normal? Can't imagine recataloging 20,000 objects. For example, if you move a thread from one NeoBoard instance to another, you have to uncatalog the thread including all its replies in NeoBoad A and catalog them in NeoBoard B: cataloging a single article object takes more than 1 second. Don't think it's normal... Or is it? - When I attempt to uncatalog an object that's not been catalogged, Zope spews out errors in the log. Can I supress the errors in code, which, in my applications, are meaningless. - Catalogs sometimes do get corrupted so recatalogging is required from time to time. Is it also normal? All of my article objects are catalog-aware and they catalog/uncatalog/recatalog themselves when getting added, deleted, or modified using manage_afterAdd(), manage_beforeDelete() and CMF'ish _edit() method. When a missing article (ghost catalog entry) causes a KeyError, NeoBoard attempts to refresh the catalog: well, takes too much time. But manually recreating its catalog is not an alternative. Any ideas why this'd happen? Any tips on maintaining catalog integrity? - Here're the indexes NeoBoard uses: security.declarePublic( 'enumerateIndexes' ) def enumerateIndexes( self ): """ Return a list of ( index_name, type ) pairs for the initial index set. """ return ( ('Title', 'TextIndex') , ('meta_type', 'FieldIndex') , ('getSortKey', 'FieldIndex') , ('getThreadSortKey', 'FieldIndex') , ('isThreadParent', 'FieldIndex') , ('creation_date', 'FieldIndex') , ('Creator', 'FieldIndex') , ('CreatorEmail', 'FieldIndex') , ('getArticleCategory', 'FieldIndex') , ('getNeoPortalContentSearchText', 'TextIndex') , ('getInlineCommentsSearchText', 'TextIndex') , ('getInlineCommentCreators', 'TextIndex') , ('getAttachmentsSearchText', 'TextIndex') , ('getNeoPortalReadCount', 'FieldIndex') , ('getNeoPortalNumContentRatings', 'FieldIndex') , ('getNeoPortalElementNumber', 'FieldIndex') , ('isTempNeoBoardArticle', 'FieldIndex') ) I came to know that 'TextIndex' is deprecated. Have yet to try ZCTextIndex or TextIndexNG ( the latter seems like an overkill). Found 'TopicIndex' very interesting. Would they make much difference? Especially, I was suprised to find the simple 'Title' index takes almost one full second when applied on an object: that getIndex( name ) call alone in the Catalog.py takes this much. So I suspect it's not about Catalog but I'm doing something very stupid in setting up this built-in catalog. ONE FINAL QUESTION: I strongly suspect I wouldn'[t be able to get any faster using ZCatalog. At least not as fast as using RDBMS. I'm thinking... "Not fast enough, not flexible enough since I can't perform sopnisticated queries on ZCatalog and stuff... why not revert to MySQL?" Got any thoughts on this? How does ZCatalog compare to a reasonably fast RDBMS? NeoBoard (1.1) will be taken out of its beta phases when I solve this catalogging weirdness, and might start working on 1.2 using MySQL or SAPDB as backend. Hope somebody can persuave me out of this path... just the thought of having to rewrite the core to use SQL makes me shudder....arrrrrrgh... Any help, hints or comments would be much appreciated. I do need to move on with this project :-( It's been almost a year now...ouch. Weeks became months; months became a whole year... whew. Thanks in advance. --------------------------------------------------------------- Wankyu Choi CEO/President NeoQuest Communications, Inc. http://www.zoper.net http://www.neoboard.net ---------------------------------------------------------------

3 5

Installing a 2nd Distro
by beno 31 Mar '03

31 Mar '03

Hi; I'm trying to install an older version of Zope in addition to my currently running version and running into difficulties. While running python w_pcgi.py Zope first complained (while compiling) that it couldn't find file /usr/local/bin/python2.2/config/Makefile.pre.in Indeed, there was no such file. But there *was* such a file here: /usr/local/bin/Python-2.2.2/Makefile.pre.in so I created a symbolic link (didn't figure it would hurt any ;) Now I get this error: Compiling extensions in lib/python cp /usr/local/lib/python2.2/config/Makefile.pre.in . make -f Makefile.pre.in boot PYTHON=/usr/local/bin/python make: *** No rule to make target `boot'. Stop. Traceback (most recent call last): File "/home/httpd/ZopeInstallation/Zope_software_home2/w_pcgi.py", line 39, in ? if __name__=='__main__': main(sys.argv[0]) File "/home/httpd/ZopeInstallation/Zope_software_home2/w_pcgi.py", line 37, in main import wo_pcgi; wo_pcgi.main(me) File "/home/httpd/ZopeInstallation/Zope_software_home2/wo_pcgi.py", line 33, in main import build_extensions File "/home/httpd/ZopeInstallation/Zope_software_home2/inst/build_extensions.py", line 30, in ? make('lib','python') File "/home/httpd/ZopeInstallation/Zope_software_home2/inst/do.py", line 62, in make do('make -f Makefile.pre.in boot PYTHON=%s' % sys.executable) File "/home/httpd/ZopeInstallation/Zope_software_home2/inst/do.py", line 32, in do if i and picky: raise SystemError, i SystemError: 512 What to do? TIA, beno

2 1

HTTP Quoting, XHTML
by Samir Mishra 31 Mar '03

31 Mar '03

Hello all, Is there a Python/Zope function for HTTP quoting of strings? Also, we're doing all of our development in strict XHTML. So far things have gone pretty smoothly. If anyone has had any prior experience, I would very much appreciate getting some idea of things to watch out for, problems, incompatibilities, etc. that I may need to watch out for. Thanks. Regards, Samir Mishra Central Bank of the U.A.E. P. O. Box 854 Abu Dhabi United Arab Emirates

3 2

RE: [Zope] Hardware for Zope + ZEO
by sean.upton＠uniontrib.com 31 Mar '03

31 Mar '03

Toby is right about using the Catalog, I think (or some other form of indexing). The ObjectManager methods are not going to scale well because: 1 - An ObjectManager/Folder has no way of batching small sets of the large store. a. BTreeFolder 2 can batch, but only in its default sort (see 2) 2 - An ObjectManager/Folder has no way of sorting, unless you do it after the fact in presentation logic (slow/bloated). You should likely be using the catalog, and create a string-based FieldIndex containing the absolute physical path of the folder, and indexes for all the fields you might sort on. Batching with a Catalog query should just be a matter of getting a slice of the returned list of Brain objects. You can write a wrapper around this that will just get the objects those Brains refer to anyway, since it is likely your application will need them. What I might suggest is writing a simple product that creates a Catalog-backed folder: it effeciently returns sorted slices of all or a subset of objects in the folder, encapsulating access to the Catalog, and creating your own API for a slightly smarter folder. So you would be adding some smart methods on top of the existing dumb methods in the ObjectManager interface that the folder already exposes. Then, each folder can have its own Catalog specifically to act as a sorting engine, and its own batching code that uses the sorting engine. I'm not sure what the performance implications of this are for your application on write (yet another Catalog to write to), but the implications on read performance should improve, given what your application does. Note: supposedly, there are eventually going to be improvements to the ZCatalog sorting code (in Zope 2.7) that will improve its performance in general by some great degree. Sean -----Original Message----- From: Wankyu Choi [mailto:wankyu@neoqst.com] Sent: Friday, January 31, 2003 3:48 AM To: 'Toby Dickenson'; 'Mario Valente'; zope(a)zope.org Subject: RE: [Zope] Hardware for Zope + ZEO > Is there a way in Zope to limit the returned > results as a limit clause would do in an SQL statement? >>The best approach is to use ZCatalog. ZCatalog means you only need to load the >>articles into memory when you need them. A threaded board needs to maintain its threaded structure. It's rather cumbersome with SQL but efficient. In Zope, it's a snap but gives poor performance. A folder (parent article) and its subfolders (its replies) strucutre directly turns into a thread hierarchy. I do use ZCatalog for all views of NeoBoard except for its default threaded look. Maybe I was stupid in not using ZCatalog for the threaded look. I'll give it a try. But I just wanted to know if there was a way to pull a limited set of objects from a **folderish object** not using ZCatalog. Not sure if your suggestion would work with NeoBoard yet, but I ran into a number of situations where I should pull all objects from a folder. I posted a simliar query months ago when I was trying to write my own user folder. No definate answer yet. The stock userfolder, for example, returns every user object no matter how many there are in response to a getUsers method call. I wouldn't use it with my 300,000 users. Nobody would, I suspect. There was no info on how to pull a limited set of users from a user folder. And guess we can't use ZCatalog to do the job. Simply put, are Zope folderish objects including BTreeFolderish objects **not** designed to hold this many objects? Well, if so, I'll just rewrite NeoBoard to use MySQL again. That would be a lot easier :( Thanks again for your time. Regards, Wankyu Choi --------------------------------------------------------------- To the dedicated staff at NeoQuest, language is not a problem to be dealt with, but an art waiting to be performed. --------------------------------------------------------------- Wankyou Choi CEO/President NeoQuest Communications, Inc. 3rd Floor, HMC Bldg., 730-14, Yoksam-dong, Kangnam-gu Seoul, Korea Tel: 82-2 - 501 - 7124 Fax: 82-2-501-7058 Corporate Home: http://www.neoqst.com Personal Home: http://www.zoper.net, http://www.neoboard.net e-mail: wankyu(a)neoqst.com --------------------------------------------------------------- _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

6 7

html form authentication
by Iñigo Serna 31 Mar '03

31 Mar '03

Hello, I want to authenticate users from acl_users with my own HTML form and a python script. I can't use user._getPassword() because it's an internal method. and it seems user.authenticate function calls browser authentication again. How could I achieve this? This is the code I use: *************** ZPT "form" *************** ... <form action="login" method="post"> <table class="form"> <tbody> <tr> <th class="text">Name: </th> <td><input name="user" size="12" maxlength="12" value=""></td> </tr> <tr> <th class="text">Password: </th> <td><input name="password" type="password" size="12" maxlength="8" value=""></td> </tr> <tr><td> </td></tr> <tr> <td align="center"> <input type="reset" value="Clean"> </td> <td align="center"> <input type="SUBMIT" value="Submit"> </td> </tr> </tbody> </table> </form> ... ********************************************* *************** Script (Python) "login" *************** request = container.REQUEST RESPONSE = request.RESPONSE user = container.acl_users.getUser(request.user) try: if user.authenticate('the_password', request): print 'OK' else: print 'FAILED' except AttributeError: print 'User doesn\' exist' return printed ********************************************* Thanks in advance, Iñigo

4 4

RE: [Zope] Zope performance documentation
by sean.upton＠uniontrib.com 31 Mar '03

31 Mar '03

You need apache or Squid in front to support virtual hosting, HTTP 1.1, possibly caching, better security (ACLS, prevent DoS). You need ZEO regardless of if you run in on 1 box or 100. In future Zope releases, ZEO will be an in-the-box sort of thing. For your traffic, you want page caching, especially for things like the PDFs you mention. Consider Squid, or at the very least Apache+mod_proxy with caching of some sort configured. For your traffic, you also need more than one ZEO client, perhaps on a single multi-processor machine (scale up), or on a farm of nodes (scale out). I can't really tell you how many. I think you need at least the following: - 1 fast box to run as a Squid or Apache caching proxy, mimimum 1-2 GB RAM and ideally a dedicated raid 0 for cache) - a few fast boxes to run Zope on - 1 box with lots of ram and fast storage to run your ZEO storage server on The last two could potentially be one big box, if done right. Sean -----Original Message----- From: Fabio Rolando [mailto:ziofunk@tiscali.it] Sent: Monday, March 31, 2003 7:10 AM To: zope(a)zope.org Subject: Re: [Zope] Zope performance documentation ...forgive me if I bother you. what would Apache do better if I have no files served outside of Zope? I thought that I'd need apache only if I have files to serve that are not "passing" through Zope.Am I wrong? Fabio For At 17:12 31-03-03 Monday, you wrote: >I would use ExtFile for the PDFs, Apache in front and ZEO for failure. > >1 GHz + Pentiums, 1 GB of RAM (Zope/Python love RAM,) SCSI drives in RAID. > >BZ > > > First of all thank you a lot for you attention. > > I'm quite new to Zope so it is still difficoult to understand how it will > > work when the system will be deployed. > > We need to size the system for (all values are the absolute maximumwe > > expect) 10.000 authenticated users, 10.000 documents (5000 pdf with 200 Kb > > average size, and 500 html), 1000 hits/minute. Do you think we can use > > Zope > > alone (no ZEO, no Apache)? > > Does the server's hardware need to carry anything special? > > Thank you. > > > > > > > >> >> Hardware is up to you and what you need. You can run it on almost > >> >> anything, but if you are going to run http://www.cbsnewyork.com you > >> >> will > >> >> need more (I think I read that all the sites combined in that Zope > >> >> installation run on 48 dual proc linux boxes). > >> > > >> > that's way off (it's less). maybe i'll ask the boss-man today to see > >> if > >> > i am allowed to publish some real figures in a document somewhere. > >> > > >> > jens > >> > > >> > > >> > _______________________________________________ > >> > Zope maillist - Zope(a)zope.org > >> > http://mail.zope.org/mailman/listinfo/zope > >> > ** No cross posts or HTML encoding! ** > >> > (Related lists - > >> > http://mail.zope.org/mailman/listinfo/zope-announce > >> > http://mail.zope.org/mailman/listinfo/zope-dev ) > >> > > >> > >> > >>_______________________________________________ > >>Zope maillist - Zope(a)zope.org > >>http://mail.zope.org/mailman/listinfo/zope > >>** No cross posts or HTML encoding! ** > >>(Related lists - > >> http://mail.zope.org/mailman/listinfo/zope-announce > >> http://mail.zope.org/mailman/listinfo/zope-dev ) > > > > > > _______________________________________________ > > Zope maillist - Zope(a)zope.org > > http://mail.zope.org/mailman/listinfo/zope > > ** No cross posts or HTML encoding! ** > > (Related lists - > > http://mail.zope.org/mailman/listinfo/zope-announce > > http://mail.zope.org/mailman/listinfo/zope-dev ) > > > > >_______________________________________________ >Zope maillist - Zope(a)zope.org >http://mail.zope.org/mailman/listinfo/zope >** No cross posts or HTML encoding! ** >(Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

1 0