March 2003 - Zope - Zope lists

HTTP Quoting, XHTML
by Samir Mishra 31 Mar '03

31 Mar '03

Hello all, Is there a Python/Zope function for HTTP quoting of strings? Also, we're doing all of our development in strict XHTML. So far things have gone pretty smoothly. If anyone has had any prior experience, I would very much appreciate getting some idea of things to watch out for, problems, incompatibilities, etc. that I may need to watch out for. Thanks. Regards, Samir Mishra Central Bank of the U.A.E. P. O. Box 854 Abu Dhabi United Arab Emirates

3 2

RE: [Zope] Hardware for Zope + ZEO
by sean.upton＠uniontrib.com 31 Mar '03

31 Mar '03

Toby is right about using the Catalog, I think (or some other form of indexing). The ObjectManager methods are not going to scale well because: 1 - An ObjectManager/Folder has no way of batching small sets of the large store. a. BTreeFolder 2 can batch, but only in its default sort (see 2) 2 - An ObjectManager/Folder has no way of sorting, unless you do it after the fact in presentation logic (slow/bloated). You should likely be using the catalog, and create a string-based FieldIndex containing the absolute physical path of the folder, and indexes for all the fields you might sort on. Batching with a Catalog query should just be a matter of getting a slice of the returned list of Brain objects. You can write a wrapper around this that will just get the objects those Brains refer to anyway, since it is likely your application will need them. What I might suggest is writing a simple product that creates a Catalog-backed folder: it effeciently returns sorted slices of all or a subset of objects in the folder, encapsulating access to the Catalog, and creating your own API for a slightly smarter folder. So you would be adding some smart methods on top of the existing dumb methods in the ObjectManager interface that the folder already exposes. Then, each folder can have its own Catalog specifically to act as a sorting engine, and its own batching code that uses the sorting engine. I'm not sure what the performance implications of this are for your application on write (yet another Catalog to write to), but the implications on read performance should improve, given what your application does. Note: supposedly, there are eventually going to be improvements to the ZCatalog sorting code (in Zope 2.7) that will improve its performance in general by some great degree. Sean -----Original Message----- From: Wankyu Choi [mailto:wankyu@neoqst.com] Sent: Friday, January 31, 2003 3:48 AM To: 'Toby Dickenson'; 'Mario Valente'; zope(a)zope.org Subject: RE: [Zope] Hardware for Zope + ZEO > Is there a way in Zope to limit the returned > results as a limit clause would do in an SQL statement? >>The best approach is to use ZCatalog. ZCatalog means you only need to load the >>articles into memory when you need them. A threaded board needs to maintain its threaded structure. It's rather cumbersome with SQL but efficient. In Zope, it's a snap but gives poor performance. A folder (parent article) and its subfolders (its replies) strucutre directly turns into a thread hierarchy. I do use ZCatalog for all views of NeoBoard except for its default threaded look. Maybe I was stupid in not using ZCatalog for the threaded look. I'll give it a try. But I just wanted to know if there was a way to pull a limited set of objects from a **folderish object** not using ZCatalog. Not sure if your suggestion would work with NeoBoard yet, but I ran into a number of situations where I should pull all objects from a folder. I posted a simliar query months ago when I was trying to write my own user folder. No definate answer yet. The stock userfolder, for example, returns every user object no matter how many there are in response to a getUsers method call. I wouldn't use it with my 300,000 users. Nobody would, I suspect. There was no info on how to pull a limited set of users from a user folder. And guess we can't use ZCatalog to do the job. Simply put, are Zope folderish objects including BTreeFolderish objects **not** designed to hold this many objects? Well, if so, I'll just rewrite NeoBoard to use MySQL again. That would be a lot easier :( Thanks again for your time. Regards, Wankyu Choi --------------------------------------------------------------- To the dedicated staff at NeoQuest, language is not a problem to be dealt with, but an art waiting to be performed. --------------------------------------------------------------- Wankyou Choi CEO/President NeoQuest Communications, Inc. 3rd Floor, HMC Bldg., 730-14, Yoksam-dong, Kangnam-gu Seoul, Korea Tel: 82-2 - 501 - 7124 Fax: 82-2-501-7058 Corporate Home: http://www.neoqst.com Personal Home: http://www.zoper.net, http://www.neoboard.net e-mail: wankyu(a)neoqst.com --------------------------------------------------------------- _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

6 7

html form authentication
by Iñigo Serna 31 Mar '03

31 Mar '03

Hello, I want to authenticate users from acl_users with my own HTML form and a python script. I can't use user._getPassword() because it's an internal method. and it seems user.authenticate function calls browser authentication again. How could I achieve this? This is the code I use: *************** ZPT "form" *************** ... <form action="login" method="post"> <table class="form"> <tbody> <tr> <th class="text">Name: </th> <td><input name="user" size="12" maxlength="12" value=""></td> </tr> <tr> <th class="text">Password: </th> <td><input name="password" type="password" size="12" maxlength="8" value=""></td> </tr> <tr><td> </td></tr> <tr> <td align="center"> <input type="reset" value="Clean"> </td> <td align="center"> <input type="SUBMIT" value="Submit"> </td> </tr> </tbody> </table> </form> ... ********************************************* *************** Script (Python) "login" *************** request = container.REQUEST RESPONSE = request.RESPONSE user = container.acl_users.getUser(request.user) try: if user.authenticate('the_password', request): print 'OK' else: print 'FAILED' except AttributeError: print 'User doesn\' exist' return printed ********************************************* Thanks in advance, Iñigo

4 4

RE: [Zope] Zope performance documentation
by sean.upton＠uniontrib.com 31 Mar '03

31 Mar '03

You need apache or Squid in front to support virtual hosting, HTTP 1.1, possibly caching, better security (ACLS, prevent DoS). You need ZEO regardless of if you run in on 1 box or 100. In future Zope releases, ZEO will be an in-the-box sort of thing. For your traffic, you want page caching, especially for things like the PDFs you mention. Consider Squid, or at the very least Apache+mod_proxy with caching of some sort configured. For your traffic, you also need more than one ZEO client, perhaps on a single multi-processor machine (scale up), or on a farm of nodes (scale out). I can't really tell you how many. I think you need at least the following: - 1 fast box to run as a Squid or Apache caching proxy, mimimum 1-2 GB RAM and ideally a dedicated raid 0 for cache) - a few fast boxes to run Zope on - 1 box with lots of ram and fast storage to run your ZEO storage server on The last two could potentially be one big box, if done right. Sean -----Original Message----- From: Fabio Rolando [mailto:ziofunk@tiscali.it] Sent: Monday, March 31, 2003 7:10 AM To: zope(a)zope.org Subject: Re: [Zope] Zope performance documentation ...forgive me if I bother you. what would Apache do better if I have no files served outside of Zope? I thought that I'd need apache only if I have files to serve that are not "passing" through Zope.Am I wrong? Fabio For At 17:12 31-03-03 Monday, you wrote: >I would use ExtFile for the PDFs, Apache in front and ZEO for failure. > >1 GHz + Pentiums, 1 GB of RAM (Zope/Python love RAM,) SCSI drives in RAID. > >BZ > > > First of all thank you a lot for you attention. > > I'm quite new to Zope so it is still difficoult to understand how it will > > work when the system will be deployed. > > We need to size the system for (all values are the absolute maximumwe > > expect) 10.000 authenticated users, 10.000 documents (5000 pdf with 200 Kb > > average size, and 500 html), 1000 hits/minute. Do you think we can use > > Zope > > alone (no ZEO, no Apache)? > > Does the server's hardware need to carry anything special? > > Thank you. > > > > > > > >> >> Hardware is up to you and what you need. You can run it on almost > >> >> anything, but if you are going to run http://www.cbsnewyork.com you > >> >> will > >> >> need more (I think I read that all the sites combined in that Zope > >> >> installation run on 48 dual proc linux boxes). > >> > > >> > that's way off (it's less). maybe i'll ask the boss-man today to see > >> if > >> > i am allowed to publish some real figures in a document somewhere. > >> > > >> > jens > >> > > >> > > >> > _______________________________________________ > >> > Zope maillist - Zope(a)zope.org > >> > http://mail.zope.org/mailman/listinfo/zope > >> > ** No cross posts or HTML encoding! ** > >> > (Related lists - > >> > http://mail.zope.org/mailman/listinfo/zope-announce > >> > http://mail.zope.org/mailman/listinfo/zope-dev ) > >> > > >> > >> > >>_______________________________________________ > >>Zope maillist - Zope(a)zope.org > >>http://mail.zope.org/mailman/listinfo/zope > >>** No cross posts or HTML encoding! ** > >>(Related lists - > >> http://mail.zope.org/mailman/listinfo/zope-announce > >> http://mail.zope.org/mailman/listinfo/zope-dev ) > > > > > > _______________________________________________ > > Zope maillist - Zope(a)zope.org > > http://mail.zope.org/mailman/listinfo/zope > > ** No cross posts or HTML encoding! ** > > (Related lists - > > http://mail.zope.org/mailman/listinfo/zope-announce > > http://mail.zope.org/mailman/listinfo/zope-dev ) > > > > >_______________________________________________ >Zope maillist - Zope(a)zope.org >http://mail.zope.org/mailman/listinfo/zope >** No cross posts or HTML encoding! ** >(Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

1 0

Help with tree tag and localfs
by Greg Fischer 31 Mar '03

31 Mar '03

I installed LocalFS, which is pretty cool because I need to add large files for users, but I need to use the dtml-tree tag to list the folders contents. When I use it normally it doesn't work: <dtml-tree LocalFSFolder branches="objectValues" sort="id"> <dtml-var id> </dtml-tree> I have tried several different options, and I have searched the internet and newsgroups, but I haven't found much. Any help is appreciated. Thanks! Greg sysman(a)techgroupinc.com

2 1

Python/Zope bindings for Aspell
by Dennis Allison 31 Mar '03

31 Mar '03

I'm looking for a Python wrapper to Aspell. Because the document to be spell checked is structured, I'd prefer an interface similar to the C++ Asepell API. Using Aspell in the pipe mode is a possibility, but direct access to the Aspell API would lead to a cleaner system.

2 2

Using CPS - utilisateurs CPS ?
by Didier Georgieff 31 Mar '03

31 Mar '03

Hello, Bonjour, - I'm doing a submission for the e-europe awards and i'd like to have a (brief) plug about what organization are actually using CPS. - Je suis en train de soumettre un dossier aux e-europe awards, et j'aimerai avoir un bref appercu des structures utilisatrices actuellement de CPS. - You can email me privately, and i'll submarize if there is some interest about it. - Vous pouvez me mailer directement et je ferai un compte rendu si cela interesse quelqu'un. - Thanks a lot in advance. - Avec mes remerciements. -- Didier Georgieff Délégation Interservice pour le Développement du SIT67 tél : 03.88.76.78.35 - fax : 03.88.76.78.34 email : didier.georgieff(a)agriculture.gouv.fr DIDSIT c/o DRAF Alsace - Cité Administrative 14, rue du maréchal Juin - 67084 Strasbourg Cedex

1 0

Referring to internal Metal Macros (in the same ZPT)?
by Johan Carlsson [EasyPublisher] 31 Mar '03

31 Mar '03

Hi, Is there an easier way to refer to a macro defined in same ZPT that calls the macro: <html xmlns:tal="..." xmlns:metal="..."><body> <div metal:define-macro="test">Test</div> <div metal:define-macro="default"> <div metal:use-macro="here/<this_zpt's_id>/macros/test"/> </div> </body></html> What I would whan it: <div metal:define-macro="default"> <div metal:use-macro="macros/test"/> </div> where macros would be the local macros. In this way I can rename the ZPT without breaking the functionality of the template. Best Regards, Johan Carlsson -- Easy Publisher Developers Team Johan Carlsson johanc(a)easypublisher.com Mail: Birkagatan 9 SE-113 36 Stockholm Sweden Phone +46-(0)8-32 31 23 Fax +46-(0)8-32 31 83 Mobil +46-(0)70-558 25 24 http://www.easypublisher.com

3 5

RE: [Zope] Zope backup
by sean.upton＠uniontrib.com 31 Mar '03

31 Mar '03

Copying a FileStorage while transactions are being appended to the end of it potentially means that the copy completes with before the transaction commit is totally flushed to the file, so there is always the possibility that your backup is going to need to have half-written transactions manually truncated to be used on either a replica or a new Zope/ZSS instance; this isn't that big of a deal, but it is possible to avoid this manual work with dirstorage. The only things that make DirectoryStorage better in this regard is that the backup tools integrate with the storage instead of acting uninformed below it, trigger snapshot mode, and get a list of files to backup from the storage software itself (this is quicker and a better guarantee than, say, using unix 'find' and mtime on a dirstorage directory to do the same thing). Compared to FileStorage, you do not have the problem of backing up files being written to because: (a) Snapshot mode prevents changes to an object in HOME/A from being written to, buffering any writes to those files in HOME/journal and HOME/B for later flush once snapshot mode is exited (post-backup). (b) Additional transactions and objects are not added to the directory being backed up. DirectoryStorage also is preferable in these backup scenarios: 1. Disaster-preparedness. You want to backup a big storage over a WAN connection - and this means incremental. You need incremental backup and IIRC something like rsync may not work very well on a changing FileStorage Data.fs. http://mail.zope.org/pipermail/zodb-dev/2002-November/003807.html We run servers at a co-location facility, and need remote backup to our facility over a 1.5Mb/s connection, and a reasonable way to do this is use the backup.py tool to create full and incremental files locally that are pulled down to remote locations via FTP on a cron job, or even better, just run the replica.py tool from our secondary location to incrementally pull down the changes (equiv. to backup.py incremental backup, but for replica purposes) over SSH connection to our other location and to tape for standard offsite backup rotations. With FileStorage, we would have to use rsync because of bandwidth constraints, and our ability to respond quickly would be impeded by the fact that we may have to manually repair the remote copy of the filestorage via truncation of half-committed transactions. 2. ZSS High-availability clustering and replication. We have an HA cluster currently using Linux-HA heartbeat, and our crude way of copying the Data.fs is via FTP for daily snapshots in the middle of the night between our primary and secondary node. This works okay (not as well as rsync would) because this application only updates most content once-daily. However, if you have a heavier-write situation, FileStorage will not be amicable to a hot-backup clustering arrangement, because cluster software will not be able to start the ZEO storage server on the backup/secondary node in the possible case of a corrupted (even slightly) filestorage copy (someone correct me if I am wrong here). The DirectoryStorage replica.py tool addresses this by providing a secure network-enabled incremental replication mechanism that ignores incoming writes (via snapshot) to guarantee consistency and isolation (in a transactional sense) for the backup operation: the backup is consistent with the state of the storage at the point in time the snapshot mode was entered (when backup started), and incoming transactions do not effect the operation of a backup because they are isolate in HOME/journal and HOME/B while stuff is copied out of HOME/A. Given this, I feel much more comfortable that I can keep a 'hot' replica on a 'hot' backup node that is ready to take over as ZSS in the case of a failure on the primary or (mainly) the need for maintenance on the primary - and I can feel comfortable that my backup/replica reflects a recent consistent record of current heavy activity. Sean -----Original Message----- From: Chris Withers [mailto:chrisw@nipltd.com] Sent: Friday, March 28, 2003 1:05 PM To: sean.upton(a)uniontrib.com Cc: jccooper(a)jcameroncooper.com; zope(a)zope.org Subject: Re: [Zope] Zope backup sean.upton(a)uniontrib.com wrote: > Though your copy may end up needing repair after the fact; backup in this > sense is not transactional. DirectoryStorage has the best answer for this > at the moment (better than FileStorage), What lead you to this belief? cheers, Chris

3 2

Fwd: [Zope-DB] (no subject)
by Charlie Clark 31 Mar '03

31 Mar '03

This was posted to Zope-DB by mistake. Charlie On 2003-03-30 at 06:46:52 [+0200], Santoshi Reddy <sreddy(a)cise.ufl.edu> wrote: > Hi folks, > > I would like to create folders in zope.We have to use the following > python code > > context.manage_addProduct['OFSP'].manage_addFile(id, > title="", file=comments) > > > Does Anyone know what OFSP is? > > Thanks, > S.

2 1