- Zope-Dev - Zope lists

serving BLOBs
by Alan Milligan 12 Oct '04

12 Oct '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I know serving large BLOBs is rather frowned up using Z2, but I have a requirement to use the zope security machinery prior to serving a blob circa 500Mb. I thought I'd just wrap it in an ExternalFile and it would simply stream to the client. In fact I was even anticipating taking advantage of chunked transfer encoding for HTTP1.1 clients. However, I'm getting out of memory exceptions thrown by the server. This would indicate that no intermediate buffer flushing is occuring at all! What am I missing? Alan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBap9OCfroLk4EZpkRAlsPAJ9oztRXnAH8rGzeLY7+hlXKkXB4yQCfQ64I OXcuPhHJmn3V+5y/HDQEN2M= =7YAX -----END PGP SIGNATURE-----

5 10

Possible Windows Service improvements.
by Mark Hammond 12 Oct '04

12 Oct '04

Hi all, I am starting to venture into the wonderful world of Zope! With the benefit of a complete lack of Zope experience, I have been able to look at the Windows service support from a fairly clean slate. However, I also realize this lack of experience means my ideas may be naive - hence I have attempted to split them into discrete issues for discrete rejection <wink>. 1) startup error redirection. I've noticed that the main Zope service driver for Windows seems to work fine when everything is setup correctly, but when things go wrong it offers no clues as to what. This is reflected in collector item 1020 ("poor error reporting on product initialisation failure under windows"). Issue 1408 ("Configuration file imports don't see INSTANCE_HOME when running Zope as a windows service"), via the referenced thread, has evidence of someone burning a day due to this. It cost me alot of time too :) I propose: Each time the child process terminates with a non-zero return code, the tail x-bytes of the child output be written to the Windows event log, where x~2k. 2) reporting of "successful start" and "backoff" strategy. A trivial startup error (eg, PYTHONPATH not set) will cause the Zope service to hopelessly retry for a number of minutes, and not respond to shutdown requests during a retry. At the moment, as soon as the service starts it reports "successful startup" to Windows. It then begins an attempt to start the child. If the child immediately fails, the code immediately begins the "backoff" strategy. This strategy appears to have 2 main purposes: * Startup may fail due to other 'services' not having yet started, so retry in the hope they become available. * The process may die due to some obscure error - restart it. On windows, assuming we install the service to depend on the "tcpip" service, I see no reason that the first reason is valid. If the process fails quickly the first time we attempt to launch it, it is almost certainly going to fail every time we try and launch it. The current strategy also means that 3rd party services could not themselves depend on the Zope service - the Zope service will report successful startup before it really has (and therefore the dependent service may itself fail). This isn't a known requirement today, but who knows! "net start" and other front ends also fail to detect fatal errors - they all say Zope started OK. I propose: We insist the child process can be created and continues to run for x seconds (where x~5). If that fails, we report an error (never reporting to Windows that we started successfully). If the child process stays alive for this period, we report success to Windows, and then use the existing backoff strategy should it die. If the machine is heavily loaded, this 5 seconds may expire before the fatal error is hit in the child - in that case, we are simply doing what we do now - using the backoff strategy to hopelessly attempt a restart - ie, a win in most cases, and no loss in the others. 3) environment setting The service process should set a number of environment variables before spawning the child - PYTHONPATH at a minimum, and according to issue #1408, INSTANCE_HOME. It already knows these values thanks to mkzopeinstance. I'm yet to determine where these values comes from for in binary build, but I see no reason not to fix this (and possibly remove whatever magic the binary does) I propose: A few trivial os.environ insertions based on the substitutions done by mkzopeinstance, before we create the child process(es). Alternatively, we create an explicit new environment we pass to CreateProcess, but I see no good reason for that.) 4) Currently, when the process is stopped, we immediately terminate the child process. This seems dangerous. We should find a way to gracefully terminate the child, and try that before we simply kill it. I propose: That someone help me work out how to do this <wink>. I've already worked out how if the service knows the username/password of a Zope administrator, but it doesn't! Sending a Ctrl_C 'signal' doesn't work without hacks to run.py (and I'm yet to confirm it will even with such hacks). I welcome any feedback on these issues. Obviously I am willing to back each of these proposals up (except 4!) with code that seems to work :) I would also welcome feedback on the best way to proceed (ie, create a new collector for each issue? thrash it out here? give up?<wink>, etc) Note that none of these issues would require a win32all/pywin32 update. If anyone was really upset by issue 1423 ("Zope 2.7.1 won't run as service under NT"), and also able to test, I'd be willing to fix it - but that *would* require a pywin32 upgrade. Tim has already kindly filled me in on that background, so it may not be trivial (ie, I would need help!) Thanks! Mark

8 19

Re: Re: [Zope-dev] xml-rpc changes between 2.7.0 and 2.7.3= ??
by Alan Milligan 12 Oct '04

12 Oct '04

Heh, Yes - would you like to lobby RedHat to change their up2date client?? I think the reason they've done this is because it's much more efficient. Not only do they not have to read the body, they don't have to parse it. Also, xml-rpc is totally screwed because it doesn't have a file type, and the normal binary limitation is 8192 bytes of parsable data between two tags :( Alan > Hi, > > Am Di, den 12.10.2004 schrieb Alan Milligan um 0:16: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Hi, > > > > I had a client that used to work on 2.7.0, and now doesn't on 2.7.3. > > > > The problem would appear to be that it's not substituting the > > xmlrpc.Response class for a GET request on a text/xml content type and > > therefore just delegating to the str() function instead of wrapping it > > in the xml-rpc xml response tags. > > > > I'm a little confused about this, as this only appears to happen in > > HTTPRequest::processInputs which didn't seem to be invoked for my > > xml-rpc call anyway ... > > > > tcpdump follows for the sceptics ... > > > > j.. > > 08:05:04.901469 IP mistress.balclutha.org.34257 > > > mistress.balclutha.org.zope: P 1:456(455) ack 1 win 32767 > > <nop,nop,timestamp 225097763 225097723> > > E.....@.@.)..............W.....x........... > > j..GET /last-bastion.net/junk/rpm/RPC2/$RHN/mail/listPackages/1 HTTP/1.1 > > Host: localhost:8080 > > Accept-Encoding: identity > > x-client-version: 1 > > x-transport-info: Extended Capabilities Transport (C) Red Hat, Inc > > (version 1.40) > > x-up2date-version: > > user-agent: rhn.rpclib.py/$Revision: 1.40 $ > > x-rhn-transport-capability: follow-redirects=1 > > x-rhn-auth-channels: ['mail', '1'] > > content-type: text/xml > > x-info: RPC Processor (C) Red Hat, Inc (version 1.44) > > Your log is a bit confusing maybe you better use tcpflow > for dumping. But what is clearly seen is the GET request. > I wonder how this ever was supposed to work since > XML-RPC requires an entidy body for the message (which is > in XML). You can compare with RFC2616 - there is no > entidy body in GET. Your client needs to use POST. > I suspect then it should work. > > Regards > Tino > >

1 0

Re: Re: [Zope-dev] xml-rpc changes between 2.7.0 and 2.7.3= ??
by Alan Milligan 12 Oct '04

12 Oct '04

> According to the CHANGES.txt there were no changes to XMLRPC. xmlrpc.py is not faulty, it's just that an HTTPResponse object is assigned to the text/xml content type rather than the correct xmlrpc.Request :(

2 1

Re: [Zope-PAS] Re: [Plone-developers] Re: Plugin for PluggableUserFolder
by Lennart Regebro 11 Oct '04

11 Oct '04

Tres Seaver wrote: > Please continue to include zope-pas(a)zope.org in this disucssion (in > fact, I would recommend trimming *all* the other groups out; interested > parties should be willing to move to the more focused list). Except, of course, it has nothing to do with Zope-PAS. But, yet, getting four of five copies of each mail is slightly annoying.

1 0

ZPublisher timing hooks ?
by Simon Michael 08 Oct '04

08 Oct '04

Hi all, I'd like to print a "this page was rendered in ... milliseconds" as you see on php sites. I think I'd get better accuracy by getting at least the start time from ZPublisher because.. actually it may not help, but I'm thinking about the case where a request generates a redirect and another request. I'd like to time the whole thing. Do you know if there are any facilities for this already in ZPublisher ? Any tips or pointers welcome. -Simon

4 3

Two problems with 2.7.3b1 and CMF1.5.0b2
by Lennart Regebro 08 Oct '04

08 Oct '04

1. Trying to install CPS on the above platform, I get the following error: ValueError: Not a registered directory: Products/CPSDefault/skins/cps_javascript (Also, an error occurred while attempting to render the standard error message.) Has anything changed with the skin directory registration? I can't find any change log to check... 2. Worse: I can't debug it. The standard import pdb; pdb.set_trace() does nothing at all! I checked (by adding prints) that the pdb and bdb code is reached, and it is, but nothing happend! And yes, debug-mode in on. Any ideas?

1 0

Zope and Eclipse WEBDAV
by Nick Bower 05 Oct '04

05 Oct '04

Hi - I'm trying to use Eclipse to connect to Zope over webdav and keep getting a 401 (unauthorised) on PROPFIND. I'm using the management user and security settings are in order. Does anyone know why this might be and if anyone has had success with Eclipse & Zope over webdav? Thanks, Nick

3 2

transience and gc.set_threshold(1)
by Michael Dunstan 05 Oct '04

05 Oct '04

I'm seeing test failures in transience when running the tests for Zope-2_7-branch (python 2.3.3) with garbage collection threshold0 set to 1. Seems to break in slightly different places from run to run. Here is one example: % python test.py -vv --dir lib/python/Products/Transience/tests -g 1 "" "testPathological" gc threshold: (1, 10, 10) Running unit tests at level 1 Running unit tests from /Zope/lib/python/Products/Transience/tests testPathologicalLeftBranching (Products.Transience.tests.testTransientObjectContainer.TestTransientObj ectContainer) ... ERROR testPathologicalRightBranching (Products.Transience.tests.testTransientObjectContainer.TestTransientObj ectContainer) ... ERROR ====================================================================== ERROR: testPathologicalLeftBranching (Products.Transience.tests.testTransientObjectContainer.TestTransientObj ectContainer) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Zope/lib/python/Products/Transience/tests/ testTransientObjectContainer.py", line 193, in testPathologicalLeftBranching map(self.t.__delitem__, revr) File "/Zope/lib/python/Products/Transience/Transience.py", line 470, in __delitem__ del bucket[k] KeyError: 946 ====================================================================== ERROR: testPathologicalRightBranching (Products.Transience.tests.testTransientObjectContainer.TestTransientObj ectContainer) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Zope/lib/python/Products/Transience/tests/ testTransientObjectContainer.py", line 183, in testPathologicalRightBranching map(self.t.__delitem__, r) File "/Zope/lib/python/Products/Transience/Transience.py", line 470, in __delitem__ del bucket[k] KeyError: 69 ---------------------------------------------------------------------- Ran 2 tests in 4.659s FAILED (errors=2)

2 2

[Zope-dev] RE: [Zope-DB] 3114, 'ORA-03114: not connected to ORACLE', DCOracle2
by Dieter Maurer 05 Oct '04

05 Oct '04

Please stay on the list... readded... Remy Pinsonnault wrote at 2004-10-4 14:39 -0400: >What do you mean had died? Oracle is quite a flexible system. We used it in a setup that each connection created a new Oracle server process. When I wrote "had died", I meant the the process responsible for one session died (going out of existence, usually due to a SIGSEGV). -- Dieter

1 0