- Zope-Dev - Zope lists

opinion: speeding up large PUT uploads
by Chris McDonough 06 Apr '05

06 Apr '05

Zope's ZPublisher.HTTPRequest.HTTPRequest class has a method named "processInputs". This method is responsible for parsing the body of all requests. It parses all upload bodies regardless of method: PUT, POST, GET, HEAD, etc. In doing so, it uses Python's FieldStorage module to potentially break apart multipart/* bodies into their respective parts. Every invocation of FieldStorage creates a tempfile that is a copy of the entire upload body. So in the common case, when a large file is uploaded via HTTP PUT (both DAV and external editor use PUT exclusively), here's what happens: - ZServer creates a tempfile T1 to hold the file body as it gets pulled in. - When the request makes it to the publisher, processInputs is called and it hands off tempfile T1 to FieldStorage. - FieldStorage reads the entire body and creates another tempfile T2 (an exact copy of T1*, in the case of a PUT request). - T2 is eventually put into REQUEST['BODYFILE']. (*) At least I can't imagine a case where it's not an exact copy. This is costly on large uploads. I'd like to change the top of the processInputs method to do this: if method == 'PUT': # we don't need to do any real input processing if we are # handling a PUT request. self._file = self.stdin return Can anyone think of a reason I shouldn't do this? - C

4 6

(Security) Hotfix_20050405 Released (URL correction)
by Tres Seaver 05 Apr '05

05 Apr '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Overview Zope Corporation has released a Zope hotfix product addressing a potential vulnerability discovered during a recent security audit of Zope 2.7 and 2.8. Affected Versions The hotfix affects versions 2.7.5 and earlier of Zope on the 2.7 release line, as well as versions 2.8a1 and 2.8a2 on the upcoming 2.8 release line. The vulnerability will be resolved in versions 2.7.6 and 2.8b1. We recommend that any site which permits untrusted users to write PythonScripts apply this hotfix, and upgrade to a fixed version of Zope as it becomes available. Further Information Please see the "product README", http://www.zope.org/Products/Zope/Hotfix-2005-04-05/Hotfix-20050405/README.… for details on the vulnerability, and for instructions on installing the hotfix. Downloading the Hotfix - "Unix tarball", http://www.zope.org/Products/Zope/Hotfix-2005-04-05/Hotfix-20050405/Hotfix_… - "Windows ZIP archive", http://www.zope.org/Products/Zope/Hotfix-2005-04-05/Hotfix-20050405/Hotfix_… Apologies for the earlier typoed URLs. Tres Seaver. - -- =============================================================== Tres Seaver tseaver(a)zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCUtIhGqWXf00rNCgRAitxAJ9Vualp5LLSrMQb1T799UWKa1UJoQCgmCJ2 EqH0Sj4RN0V8o1ldX6C1g90= =1lBU -----END PGP SIGNATURE-----

1 0

(Security) Hotfix_20050405 Released
by Tres Seaver 05 Apr '05

05 Apr '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Overview Zope Corporation has released a Zope hotfix product addressing a potential vulnerability discovered during a recent security audit of Zope 2.7 and 2.8. Affected Versions The hotfix affects versions 2.7.5 and earlier of Zope on the 2.7 release line, as well as versions 2.8a1 and 2.8a2 on the upcoming 2.8 release line. The vulnerability will be resolved in versions 2.7.6 and 2.8b1. We recommend that any site which permits untrusted users to write PythonScripts apply this hotfix, and upgrade to a fixed version of Zope as it becomes available. Further Information Please see the "product README", http://www.zope.org/Products/Zope/Hotfix-2005-04-05/Hotfix-200405/README.txt for details on the vulnerability, and for instructions on installing the hotfix. Downloading the Hotfix - "Unix tarball", http://www.zope.org/Products/Zope/Hotfix-2005-04-05/Hotfix-200405/Hotfix_20… - "Windows ZIP archive", http://www.zope.org/Products/Zope/Hotfix-2005-04-05/Hotfix-200405/Hotfix_20… Tres Seaver. - -- =============================================================== Tres Seaver tseaver(a)zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCUsvGGqWXf00rNCgRAt3qAJ42sH4BIPP9+S1g+ZnpwS9YopcggQCfYnvw hXfT3SOxuL1y1adv5zmv3v8= =smRT -----END PGP SIGNATURE-----

1 0

[Zope2.8a2] ...to be released by tomorrow....
by Andreas Jung 04 Apr '05

04 Apr '05

Hi, I am planning to make the release tomorrow (Saturday afternoon (German time :-)). So please make your final fixes very soon or cry out loud STOP if there are any show stoppers. Andreas

4 12

post publishing hook
by Florent Guillaume 04 Apr '05

04 Apr '05

I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times. Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying. I'd like a post-publishing hook that's called in the initial REQUEST and transaction. Open issues are: 1. what if there's an exception in the hook? I'd say log it but that's all. 2. what if there's a Conflicterror? That's tougher to deal with... The transaction has to be retried, without the user being aware of it, so it means the hook should probably be called *before* things are sent to the browser. What do you think? Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

7 15

HEADS UP: QA Problems with Zope 2
by Christian Theune 04 Apr '05

04 Apr '05

Hi, as I'm the guy who clicks the "build" button for the Zope 2 Windows releases I have the unfortunate honor to write his complaining mail: Situation ========= a) Zope 2.8 as of release a2 does not run bin/test.py anymore. Neither on Linux nor on Windows. Unfortunately this is the only testrunner worked on Windows at all. Now it is broken as well. (Import problems from Five.products or something) b) The WinBuilder environment isn't version managed along the Zope branches. The change of the Zope module to Zope2 required manual changes in the WinBuilder structure that broke and didn't got detected during continous tests. (See c) Additionally the requirements of specific versions Python, ZODB, winutils etc. are hardcoded in the Makefiles right now. c) There are no nightly tests for Windows at all. I know there is some suspended effort already around. Please notice this as a friendly reminder that I'd be very lucky having those catch errors *before* Andreas makes a release and I run the tests manually. d) zopectl on windows doesn't work (it barfs about SIG_CHILD not existing). I'm not sure if it ever did or if it should. If it should not work on windows anyway, maybe we should remove it in the windows distribution ... All in all, it's a very unsatisfying situation for the Windows users, and I'm pretty scared telling anybody about the situation when I'm asked. This really has to change at least a bit. What to do ========== Things I can do and propose to do to make this better: ------------------------------------------------------ b): I can either create branches for WinBuilders responding to the Zope Versions. Or (what I like better) I can put the WinBuilders somewhere in the Zope 2 tree to allow versioning along a branch automatically so continuous tests know where to get the WinBuilders from. c): If there is some infrastructure with the build bot around already and someone gives me pointers, I can set up nightly builds and tests on a Win2k machine over here. Additionally the tests should run on XP Server (or professional) and Win 2003 server as well. I don't have those around unfortunately + I don't have enough Visual Studio licenses around. Anyway, having at least one continuously running test is better than having none. Things I can't do anything right now on my own and need support: ---------------------------------------------------------------- a) I'm not able to look into the test runners (that are not broken solely on windows) nor to look into Five integration problems. I'd love if someone could a) either fix test.py or b) deprecate it and give me the hint to some other runner that works. test.py is included and should therefore work within a release or be removed. I'm not sure what is right here. d) I have no idea about zopectl on windows. Is there some knowledge around on this? ================ Final conclusion ================ I have a compiled Zope 2.8a2 around here, but I'm not able to run the unit tests on it and I'm not willing to publish it therefore because I have the suspicion that this branch never has seen windows before except the one pass of unit tests before 2.8a1. Cheers, Christian -- gocept gmbh & co. kg - schalaunische str. 6 - 06366 koethen - germany www.gocept.com - ct(a)gocept.com - phone +49 3496 30 99 112 - fax +49 3496 30 99 118 - zope and plone consulting and development

5 7

OS X 10.3.8 + mount_webdav + Zope all on powerbook
by Pavel Zaitsev 02 Apr '05

02 Apr '05

Good evening(or morning?);-), I am just wondering if any of you have such installation, I am having problems with OS X because of Zope behaviour of HTTPRequest and cgi.py it uses for FieldStorage which in turn uses TemporaryFile. I need responses only from people that have OS X 10.3.8, zope 2.7+ mounted via mount_webdav... I wish to cross notes before nuking my installation and redoing the install, testing for the bug and filing it with apple. Problem is that when using mount_webdav agains zope, mount_webdav is much too verbose and requests status for every entity in the directory, invoking HTTPRequest and thus creating at least one temporary file per listed. When sporradically browsing the mount point, zope comes in conflict with some internal locks on the filesystem access that webdav places... that is speculation from my experience. Open call never returns, I have made tracebacks when the system stalls. Zope administrator process kill the instance that does not respond to however it is being queried being alive .. but its gone into the kernel and no threads are running, so it is killed off and new instance of zope satisfies the webdav request minuetes later ... everything comes back to life... I think I will modify cgi.py and release a patch so it does not use disk based files maybe just pseudo memory files. So please post if you run zope and os x with webdav mount in tandem please... much thanks, pavel

2 2

Re: [Zope-Coders] RE: [Zope-dev] [Zope2.8a2] ...to be released by tomorrow....
by Tim Peters 02 Apr '05

02 Apr '05

[Andreas Jung] > I am planning to make the release tomorrow (Saturday afternoon > (German time :-)). So please make your final fixes very soon or cry out loud > STOP if there are any show stoppers. Barring an unexpected heart attack, an alpha-quality ZODB will be in place for you by then. Jim will not be able to merge his ZClasses changes in, so those will have to wait for a later release. All the substantive changes I needed to make were completed last night, and the overnight test runs were successful. I still need to write up ZODB NEWS, fiddle the version numbers, make a 3.4a1 release tag, and edit the svn:externals in Zope trunk to point at that tag. That's all very low risk, so just assume it's been done if I don't say anything else later.

2 2

Parrot
by Alan Milligan 01 Apr '05

01 Apr '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Parrot 0.1.2 was released a couple of weeks ago. According to the release notes, there's now "better python support, separated into dynaclasses". Unfortunately, I haven't really managed to keep up with the Parrot project, but I think it quite possibly represents the strategic future of Zope. An obvious benefit is the genuine concurrency it offers over GIL on SMP hardware. Anyone running an enterprise Zope or Plone installation would desire to take advantage of this. I know a year or so ago, someone received a pie in the face at a conference because Parrot still wasn't as fast as the native Python interpreter, but maybe this too has changed with this release. I'm very interested to know if anyone is investigating Parrot. Alan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFCO4vpCfroLk4EZpkRAsVVAKCQBlffkkhd86BX/KxwJcYSXPx5pwCfdiU8 Ybnu4zF97xG8SSYxYUY64I8= =KzS4 -----END PGP SIGNATURE-----

5 7

editing zope files
by Pavel Zaitsev 01 Apr '05

01 Apr '05

I try to edit zope file in zope istalled directory, HTTPServer in particular, i want to debug some things, but I get syntax errors on any code i try insert like a=1 , or 'print'. like: # /opt/web/bin/runzope </opt/zope_core/lib/python/Zope/Startup neo[5135] Traceback (most recent call last): File "/opt/zope_core/lib/python/Zope/Startup/run.py", line 50, in ? run() File "/opt/zope_core/lib/python/Zope/Startup/run.py", line 18, in run opts = _setconfig() File "/opt/zope_core/lib/python/Zope/Startup/run.py", line 42, in _setconfig opts.realize(doc="Sorry, no option docs yet.") File "/opt/zope_core/lib/python/zdaemon/zdoptions.py", line 271, in realize self.load_schema() File "/opt/zope_core/lib/python/zdaemon/zdoptions.py", line 312, in load_schema self.schema = ZConfig.loadSchema(self.schemafile) File "/opt/zope_core/lib/python/ZConfig/loader.py", line 37, in loadSchema return SchemaLoader().loadURL(url) File "/opt/zope_core/lib/python/ZConfig/loader.py", line 71, in loadURL return self.loadResource(r) File "/opt/zope_core/lib/python/ZConfig/loader.py", line 137, in loadResource schema = ZConfig.schema.parseResource(resource, self) File "/opt/zope_core/lib/python/ZConfig/schema.py", line 43, in parseResource xml.sax.parse(resource.file, parser) File "/usr/lib/python2.3/site-packages/_xmlplus/sax/__init__.py", line 31, in parse parser.parse(filename_or_stream) File "/usr/lib/python2.3/site-packages/_xmlplus/sax/expatreader.py", line 109, in parse xmlreader.IncrementalParser.parse(self, source) File "/usr/lib/python2.3/site-packages/_xmlplus/sax/xmlreader.py", line 123, in parse self.feed(buffer) File "/usr/lib/python2.3/site-packages/_xmlplus/sax/expatreader.py", line 216, in feed self._parser.Parse(data, isFinal) File "/usr/lib/python2.3/site-packages/_xmlplus/sax/expatreader.py", line 312, in start_element self._cont_handler.startElement(name, AttributesImpl(attrs)) File "/opt/zope_core/lib/python/ZConfig/schema.py", line 119, in startElement getattr(self, "start_" + name)(attrs) File "/opt/zope_core/lib/python/ZConfig/schema.py", line 306, in start_import src = self._loader.schemaComponentSource(pkg, file) File "/opt/zope_core/lib/python/ZConfig/loader.py", line 154, in schemaComponentSource __import__(package) File "/opt/zope_core/lib/python/ZServer/__init__.py", line 36, in ? from HTTPServer import zhttp_server, zhttp_handler File "/opt/zope_core/lib/python/ZServer/HTTPServer.py", line 79 print ^ How can I rectify this? pavel

1 1