- Zope - Zope lists

Scripts, external methods and security
by Adrian Blockley 18 Jun '02

18 Jun '02

Hi All, We are using Zope to publish some air quality data for our part of the world. We have written a number of simple Zope python scripts and external methods to access the air quality data. I have to confess we are all relative newbies when it comes to Zope and Python. One of the things I need to check is what level of script hardening and security measures do we need to take. We have a bit more experience with perl CGI scripts and have done standard hardening such as filtering out metacharacters etc. Do we need to take similar measures with Zope python scripting? What other security measures do we need to take? All hints and suggestions on practical security measures are greatfully appreciated. Cheers Adrian -- ======================================================================= Adrian Blockley (adrian.blockley(a)environ.wa.gov.au) Western Australian Department of Environment Water and Catchment Protection Address: Department of Environment Water and Catchment Protection 141 St Georges Tce Perth WA 6000 Ph: +61 8 9222 7165 Fax: +61 8 9321 5184 ======================================================================= DISCLAIMER: This e-mail is confidential to the addressee. If you are not the addressee, please notify the Department by return e-mail and delete the message from your system. You must not disclose or use the information contained in this email in any way. Thank you.

3 2

Errorno 2
by oliver.erlewein＠sqs.de 18 Jun '02

18 Jun '02

Hi all I'm experiencing a problem, wihich I can't pinpoint. I've written a small external python script, which writes a logline into a sequential file. Nothing komplex. This gets called whenever a webpage gets called. Now I get the following error: IOError, Errno 2: File or directory not found: <filename>.log The thing is this happens when I have low traffic on the Z-box. The system is up an running for about 12 months now and it has happened twice in the last 14 days. The logfile is about 330Mb. Any ideas where I would have to start looking? Thanx Oliver

2 1

How does DTML handle return variables from Python
by McDonnell, Larry 18 Jun '02

18 Jun '02

Hi, I sorted of asked this question before but I am still hitting a wall. I am using this python code: context, container, script, traverse_subpath part_num = st1 + sub_type2 return part_num But when "part_num" is returned it is blank. I test the script and "part_num" has a return value. The DTML doc is as follows: <form name="form1" method="post" action="create_part_num"> <body bgcolor="#FFFFFF" text="#000000"> <p>Your here!!<dtml-var st1><dtml-var sub_type2><dtml-var part_num> pn</p> Your here!!<dtml-var st1><dtml-var sub_type2><dtml-var part_num> pn <dtml-var expr="concat_sub_types_to_part_num(st1,sub_type2)"> **<dtml-var part_num>pn <input type="submit" name="Submit" value="Submit"> </form> To make sure I have a value for part_num on the previous dtml doc I hard coded "zz" into the variable. Also, I wanted to see it on the next dtml doc called. So I try to figure this out between network problems. Any help would be appreciated, thanks. I am a networking guy who they asked to web program. You know they asked for volunteers and everyone step back but me. Larry McDonnell Proton Energy Systems 50 Inwood Rd. Rocky Hill, CT 06067 (860) 571-6533 ext. 531 Email:lmcdonnell@protonenergy.com www.protonenergy.com

2 1

Dynamically create DTML Document
by John "Amazn" Hazen 18 Jun '02

18 Jun '02

To anyone who knows their Zope, Within Zope I'd like to create a News Submission Form such that a user can type in a title and an article. The title of the DTML Document would then correspond to the title entered by the user and the body of the DTML document would be the article entered by the user. Anyone know though how I can dynamically create this DTML Document using Zope? Thanks, -john-

2 1

Serious write-performance issue with multiple threads.
by Arnar Lundesgaard 18 Jun '02

18 Jun '02

Hi, For some time now we have had trouble with write performance on two ZOPE sites we have in operation. The first one has been in production since early April, while the second is being opened to the public in a few days. They are both based on our internally developed TopicMap engine (built on top of CMF). (To be OpenSourced when we have some available time to finish it up.) The problem has been scarily easy to reproduce, and manifests itself whenever multiple persons are working (writing articles etc). As long as only one person is working on the site, it has the expected performance. Not very fast, a few seconds per write operation, but acceptable. However as soon as more people log in and start publishing, we often experience hangs which may last for 10-15 minutes on write operations that usually takes only a few seconds to perform. In this situation Zope doesn't respond to any requests. Fortunately visitors to the first site are served by the cache (squid on the same machine) and reads from ZOPE have ranged from very fast to mostly quickly. Although we do not have any other ZOPE sites with the same write-load as these sites, the TopicMap engine has naturally been our main suspect. (Load on the first site is about the same as 'zope.org' but with much higher peaks.) It seemed obvious that something was/is with the design of our TopicMap engine that triggers this. We have been reviewing our own code and made many optimizations that we expected to yield significant speedups. While they did make the site faster on average, write operations still often result in long hangs when multiple users were working on either of the sites. We followed the procedure in http://www.zope.org/Members/4am/debugspinningzope. The hang is indeed a spinning process (thread) as it always uses 99.9% CPU when Zope stops responding; attaching to the main process in a hang situation and looking at the responsible thread invariably shows it to be in chunk_free() in libc somewhere downstream from pickleCache. Zope will usually "unhang" itself after about 10-15 minutes of spinning. Mr. Kromers comments on ZOPE and SMP in the 'system requirements' thread a couple of weeks ago gave us a few clues, which we followed. We tried binding the ZOPE instance to one CPU using the affinity patch for Linux 2.4, but that did not help either. We then tried disabling one CPU, suspecting SMP trouble, but still no go. So slowly ZOPE became the suspect. Tried different number of threads, 20, 10, 4, 2... then we we tried running with only one thread, and the "hang" problems vanished! What's more; finally we were getting the performance we expected. It works very well now, even under relatively high load and a lot of write activities. The hardware and software of the two sites: Site 1: Site 2: A dual Xeon 1.2 GHZ A dual Xeon 1.0 GHZ 2 GB ram 512 MB ram RAID 1+5 RAID 1+5 CVS ca. ZOPE 2.5.0 ZOPE 2.5.1 CVS ca. CMF 1.2 CMF 1.2 Python 2.1.2 (PThreads) Python 2.1.3 Linux 2.4.14 (XFS) Linux 2.4.18 (ext3) Python is compiled with thread support and large file support. for both sites The database for site 1 is approx 450 MB newly packed - ca. 220 000 objects; site 2 isn't live yet and is much smaller (~100MB packed). We're experiencing the same problems on both sites. The second site will be going live in a few days. Due to its design and requirements we are not able to cache as much content. This forces us to look for a different solution; the one-thread option won't cut it; too many requests will have to go outside the cache... Currently we are running this on a single thread, but we expect that to "kill us" once it is opened to the general public :( A possible workaround is of course to run two different ZOPE instances with a ZEO backend. One with multiple threads for reads and visits, the other with a single thread for writing and publishing. This is perhaps the ideal solution, but we are loath to make untested changes to the production environment just before going live. Secondly this feels like a bug in ZOPE or Python, and if it is we would like to track it down. What we are looking for is information on threading in Python/ZOPE/ZODB, other peoples experiences, workarounds, etc. Regards, Arnar Lundesgaard ---------------------------------- phone: (+47) 982 38 036 mailto:arnar.lundesgaard(a)creuna.no Creuna as Bryggegata 3 NO-0250 Oslo phone office: (+47) 23 23 88 00 fax: (+47) 23 23 88 50 http://www.creuna.no/

3 2

External Editor on Win32 -- downloading files with .html extension
by Grégoire Weber 18 Jun '02

18 Jun '02

Hi Casey, cc: hi All (on the list), I like your ExternalEditor product! It's really cool! I'am working under Win2k with Zope 2.5.1 and ExternalEditor 0.3. I have some problems editing object externaly. I could not solve them by reading the documenttaion or the mailing list. Netscape 4.79 ------------- With NS 4.79 it works as expected as long as zope objects do not have well known extensions like 'html'. I have to set the 'html' extension on objects because I'm working also with dreamweaver (via ftp/WebDAV). What happens if I click on the pencil icon of an object **with** .html as extension: NS 4.79 opens a alert box asking if the file has to be opened with mozilla.exe or saved to disk. (mozilla is my default browser for html-files) Seems to be a browser problem. The http headers of a file with or without a html extension are the same (I looked at them with tcpwatch). NS 4.79 Config: Application (correct IMO) Extension: ZOPE MIME type: application/x-zope-edit Handled by Application: d:\prog\Python22x\pythonw.exe D:\prog\ZopeEdit\zopeedit.py "%1" "%s" Mozilla ------- With Mozilla (1.0rc1) It doesn't even work to edit files without extensions. Instead I get the follwoing message box (it seems to be a mozilla box): Opening login_form.html <temp path>/login_form.html could not be opened, because an unknown error occured. Sorry about that. Try saving to disk first and then opening the file. The file containing the authentification information can be found in the temp directory (wasn't cleared). Mozille 1.0rc1 Config: Helper Applications (correct IMO) Extension: zope MIME type: application/x-zope-edit Handled by Application: d:\prog\Python22x\pythonw.exe D:\prog\ZopeEdit\zopeedit.py "%1" If you send me some instructions I woudl assist in finding the problems. Greetings, Greg ------------------------------------------------------------------------------- ExternalEditor ini file: # Zope External Editor helper application configuration [general] # General configuration options # Uncomment and specify an editor value to override the editor # specified in the environment editor = d:\prog\pfe32\pfe32.exe /m # editor = d:\winnt\system32\notepad.exe # works! # Automatic save interval, in seconds. Set to zero for # no auto save (save to Zope only on exit). save_interval = 1 # Temporary file cleanup. Set to false for debugging or # to waste disk space. Note: setting this to false is a # security risk to the zope server cleanup_files = 1 # Use WebDAV locking to prevent concurrent editing by # different users. Disable for single user use or for # better performance use_locks = 1 # Specific settings by content-type or meta-type. Specific # settings override general options above. Content-type settings # override meta-type settings for the same option. [meta-type:DTML Document] extension=.dtml [meta-type:DTML Method] extension=.dtml [meta-type:Script (Python)] extension=.py [meta-type:Page Template] extension=.pt [meta-type:Z SQL Method] extension=.sql [content-type:text/*] extension=.txt [content-type:text/html] editor = d:\prog\pfe32\pfe32.exe /m # this was only a try - didn't help [content-type:text/html] extension=.html [content-type:text/xml] extension=.xml [content-type:image/*] editor = D:\prog\PaintShopPro6\Psp.exe [content-type:image/gif] extension=.gif [content-type:image/jpeg] extension=.jpg [content-type:image/png] extension=.png _____________________________________ Grégoire Weber mailto:gregoire.weber@switzerland.org

3 4

Re: [Zope] Dreamweaver MX rewrites ZPT XML?
by hpinson＠indepthl.com 18 Jun '02

18 Jun '02

> I am not familiar with Windows Controller, but > it's probably passing arguments to Z2.py the same way start.bat > would. Could be. I think the Zope Windows Controller NT service is some compiled form of the start.bat. I wish I had never installed that Windows Controller, and it has no uninstaller (frown). It kills the default Zope service in favor of its own. It's a real black box. Have to spend some time with its documentation I guess. I hope there is some! >> Array David, you are right about the array-- Zope won't even start on W2K if you specify the WebDAV source port in array brackets, like in the example. >> Dreamweaver MX This is really an improvement over previous versions-- I hope I can get it to deal with ZPT. Otherwise the WebDAV problems of previous versions seem to have been fixed. It still will not see extensionless objects-- at least as far as I have determined. Harlow Pinson Indepth Learning Email: hpinson(a)indepthl.com Web: http://www.indepthl.com Voice: 505-994-2135 FAX: 208-475-7678

1 0

Zope on Solaris
by Dennis Heltzel 18 Jun '02

18 Jun '02

Update! It's up and running fine on Solaris 8. Converting it to a zip file did the trick. If you are interested in creating a zip file for downloading, here are the Linux commands I used: tar xvzf Zope-2.5.1-solaris-2.6-sparc.tgz zip -ry Zope-2.5.1-solaris-2.6-sparc.zip Zope-2.5.1-solaris-2.6-sparc The resulting file, Zope-2.5.1-solaris-2.6-sparc.zip, was slightly larger (8480279 bytes) than the .tgz file, but so much easier to install on a stock Solaris system. I understand that Solaris 9 will include a lot of GNU tools, and all I can say is "it's about time". Thanks for your help with this. Dennis

1 0

Zope for Solaris
by Dennis Heltzel 18 Jun '02

18 Jun '02

I tried several of the Zope Solaris binaries from your site and they all give me a directory checksum error during the "tar xvf" part of the install (they gunzip just fine). If this is a general problem, you might want to at least remove the files from your site. BTW, the Linux binaries work great, and I am quite impressed with Zope so far. If I convince my company to deploy it as part of our intranet portal, I will probability need to get it running on Solaris 8. Dennis

5 4

ANNOUNCE: Pound - reverse proxy and load balancer - v0.2
by Robert Segall 18 Jun '02

18 Jun '02

This is to announce the release of Pound v0.3. Changes from previous version: - added HTTP/1.1 handling, resulting in significant speed-ups. - added HTTP/1.1 to HTTP/1.0 proxying: if the client is 1.1 compliant, Pound will accept multiple requests per connection, while dealing with the need to re-open connections to a back-end HTTP/1.0 server. - fixed a few minor bugs. Many thanks to Ken Lalonde ken(a)torus.ca for his very helpful suggestions. What is Pound (from the Web page) - a reverse-proxy: it passes requests from client browsers to one or more back-end servers. - a load balancer: it will distribute the requests from the client browsers among several back-end servers, while keeping session information. - an SSL wrapper: Pound will decrypt HTTPS requests from client browsers and pass them as plain HTTP to the back-end browsers. - an HTTP/HTTPS sanitizer: Pound will verify requests for correctness and accept only well-formed ones. - an HTTP/1.1 to 1.0 proxy. For further information, download, etc. please see http://www.apsis.ch/pound The software is at version 0.3. Further testing (especially under heavy loads), improvements and suggestions are welcome. Pound is distributed under the GPL. Enjoy and let me know how it works for you. -- Robert Segall Apsis GmbH Postfach, Uetikon am See, CH-8707 Tel: +41-1-920 4904

1 0