- Zope-Dev - Zope lists

[Bug] FileStorage packing looses data
by Dieter Maurer 13 Apr '03

13 Apr '03

ATT: cross post We have a nasty FileStorage packing bug <http://collector.zope.org/Zope/875> The bug is caused by a strange handling of backpointers during pack's copying phase: the code checks whether the backpointer goes to a packed or unpacked position. When it goes to a packed record, it guesses what new value the backpointer should have -- and it guesses wrong. This can lead to data loss! Currently, "pack" treats its two phase quite asymmetrically with respect to position resolution: Packing phase: backpointers are resolved "prev" and "nvprev" positions are determined via "index/nvindex" Copying phase: backpointers are not resolved but adjusted depending on whether they point to packed or unpacked records "prev" and "nvprev" are only adjusted via "index/nvindex" when they point to packed data. Otherwise they are adjusted via a constant offset. I do not see any reason why this should not be symmetrical: all backpointers resolved all positions determined via "index/nvindex" No more guessing, no longer an offset that needs to be held constant. What is correct for packed data should be correct for not yet packed data, as well. Do you see an error in this argument? Dieter

4 3

Re: [Zope-dev] urgent: string exceptions
by Chris McDonough 11 Apr '03

11 Apr '03

That'd be fine, but it would require a code audit. And third party products might fail. On Fri, 2003-04-11 at 09:14, Christian Theune wrote: > Sorry for split-post. > > OTOH as the string can't be caught anyway, and it is defined as a module > variable, we could simply drop the space, hmm? > > Cheers, > Christian > > Am Fre, 2003-04-11 um 15.07 schrieb Chris McDonough: > > You can catch most string exceptions. "Bad Request" cannot be caught > > because (afaik) the string isn't interned because it has a space in it. > > > > As a workaround, I'd use except: then do > > > > try: > > something > > except: > > e = sys.exc_info() > > if e[0] == 'Bad Request': > > do something > > raise > > > > On Fri, 2003-04-11 at 05:57, Lennart Regebro wrote: > > > Christian Theune wrote: > > > > a) Remove "Bad Request" beeing a String exception and make it a class > > > > exception > > > > > > Yup. Don't use String Exceptions. One reason is that you can't catch > > > them. :) > > > > > > > > > > > > _______________________________________________ > > > Zope-Dev maillist - Zope-Dev(a)zope.org > > > http://mail.zope.org/mailman/listinfo/zope-dev > > > ** No cross posts or HTML encoding! ** > > > (Related lists - > > > http://mail.zope.org/mailman/listinfo/zope-announce > > > http://mail.zope.org/mailman/listinfo/zope ) > -- > Christian Theune, gocept gmbh & co. kg > http://www.gocept.com - ct(a)gocept.com > fon: 03496 3099112 fax: 03496 3099118 mobile: 0179 7808366 >

2 1

Re: [Zope-dev] urgent: string exceptions
by Chris McDonough 11 Apr '03

11 Apr '03

You could use ModuleSecurityInfo (see developer's guide) to put this code in a module and import from the python script. I'm not sure if we'd be able to safely expose import of the bad request exception directly from OFS, this might require some refactoring. On Fri, 2003-04-11 at 09:13, Christian Theune wrote: > This won't work within python scripts. Or am I wrong? > > Cheers, > Christian > > Am Fre, 2003-04-11 um 15.07 schrieb Chris McDonough: > > You can catch most string exceptions. "Bad Request" cannot be caught > > because (afaik) the string isn't interned because it has a space in it. > > > > As a workaround, I'd use except: then do > > > > try: > > something > > except: > > e = sys.exc_info() > > if e[0] == 'Bad Request': > > do something > > raise > > > > -- > Christian Theune, gocept gmbh & co. kg > http://www.gocept.com - ct(a)gocept.com > fon: 03496 3099112 fax: 03496 3099118 mobile: 0179 7808366 >

1 0

Re: [Zope-dev] urgent: string exceptions
by Christian Theune 11 Apr '03

11 Apr '03

I'm talking about OFS.ObjectManager, I'm sorry that I didn't state that clearly before. :) Am Fre, 2003-04-11 um 14.47 schrieb Lennart Regebro: > Christian Theune wrote: > > Problem is: I would need some module i could import that exception > > from into the python script. at least telling me which module that > > (w|c)ould be would bring me forward. > > Sorry, I don't follow you here.... > You would need to change the source of the module that raises teh > exception. That would only be a problem if you don't have the source, of > if it's in C... > > And you should also send the code back to the author with the fixes, and > kick his butt. :) -- Christian Theune, gocept gmbh & co. kg http://www.gocept.com - ct(a)gocept.com fon: 03496 3099112 fax: 03496 3099118 mobile: 0179 7808366

1 0

Re: [Zope-dev] urgent: string exceptions
by Christian Theune 11 Apr '03

11 Apr '03

Problem is: I would need some module i could import that exception from into the python script. at least telling me which module that (w|c)ould be would bring me forward. thanks Am Fre, 2003-04-11 um 13.27 schrieb Lennart Regebro: > Christian Theune wrote: > > Urks. It's not like *I* throw them ... > > Yeah, I suspected that since otherwise you would just have fixed it > already. :) > > Well, it's easy. Do a search for 'Raise "' and wrap the string in > 'Exception()' and your done. If you are really clever with sed (I'm not) > you can even to it automatically. :D -- Christian Theune, gocept gmbh & co. kg http://www.gocept.com - ct(a)gocept.com fon: 03496 3099112 fax: 03496 3099118 mobile: 0179 7808366

1 0

Streaming Response
by Danny W. Adair 11 Apr '03

11 Apr '03

# ################################################################################################ # I first posted this to zope(a)zope.org, without success. Although it is not about the development of # Zope itself, it deals with its internals, so now I post it here. # I hope everyone is alright with that. :-) # ################################################################################################ Hi, I would like to show status messages while calling a couple of functions from within a Python Script: While the script is running, I want to successively output messages like ------------------------ Doing this...ok Doing that...ok Done ------------------------ and then either redirect or display a button to move on. 1. I do _not_ want to use JavaScript 2. I do _not_ want to use the meta tag "refresh" I would like to use Response.write() to push my status messages on the fly. Since I don't know which function calls will succeed and which will fail, I do not know "Content-Length" in advance. Therefore, I would like to use a streaming/chunked response as specified in HTTP 1.1. How do I do this in Zope? I tried to do "response.setHeader('Content-Type', 'multipart/x-mixed-replace')" but couldn't get it working. (Also, it looks as if this will only work in Netscape but not IE. True? Then unfortunately I can't go that route anyway.) I also tried to leave the header alone and just "Response.write()" the multipart content-type and its boundary and the content-type's of the parts, but ZPublisher just returned the plain text... I guess the answer lies somewhere in "Transfer-Encoding: chunked" but I can't figure out how to actually do it. :-) Maybe one of you guys working on HTTPResponse.py (Fred, Chris, Brian, Toby, Andreas) knows how to accomplish this? Anyone else? I appreciate the smallest hint. ;-) Thank you very much for your help, Danny P.S.: Please include my address in your reply ("Reply All") since I am currently not subscribed to this list. Thank you.

5 7

Any chance for a 2.7 alpha in the near future?
by Craeg Strong 09 Apr '03

09 Apr '03

Howdy: We have a big project starting soon for the US government and would love to start off with 2.7 and all the python 2.2 goodies. Can anybody tell me what the latest thinking is for the roadmap? gracias, --Craeg

2 1

LocalFS/Zope/Python leaking memory on large file uploads
by John Ziniti 09 Apr '03

09 Apr '03

I am using Zope 2.4.3, Python 2.1.2 and LocalFS (version?) on both Linux and Solaris and I am trying to upload large files (~50-150 MB). Zope consumes memory to the point of strangling the machine, and I am forced to restart it. The Solaris machine has 256 MB RAM/512 MB swap. The Linux box has 512MB/1GB. I have read: http://www.zope.org/Members/tseaver/Projects/HighlyAvailableZope/BleedingLi… Does anyone have any additional advice on where to start looking for/fixing the problem? I will most likely spend most of tomorrow trying to fix the problem, and will obviously contribute anything I find back to the list/community; so any pointers/shortcuts/anecdotes that might save me some time would be appreciated. John Ziniti

2 1

Basic hassle with manage_changeProperties
by Jean Jordaan 09 Apr '03

09 Apr '03

Hi all I've encountered a very simple problem with manage_changeProperties as defined in Zope-2.6.1-src/lib/python/OFS/PropertyManager.py The signature is: def manage_changeProperties(self, REQUEST=None, **kw): and the docstring promises: Change object properties by passing either a mapping object of name:value pairs {'foo':6} or passing name=value parameters However, if you call something.manage_changeProperties({a:b}), then REQUEST will be {a:b} and the final check in manage_changeProperties will be true, and manage_propertiesForm will be returned: if REQUEST: message="Saved changes." return self.manage_propertiesForm(self,REQUEST, manage_tabs_message=message) Now, manage_propertiesForm expects REQUEST to be a proper REQUEST object and fails with a KeyError on URL1, when it is a simple dictionary. manage_changeProperties as defined in Zope-2.6.1-src/lib/python/OFS/PropertySheets.py doesn't suffer this problem, since it returns MessageDialog, which doesn't expect anything beyond the keyword parameters it's passed. Either Zope-2.6.1-src/lib/python/OFS/PropertyManager.py should be taught to distinguish between {a:b} and REQUEST, or the docstring should be changed to: Change object properties by passing either the REQUEST, or passing name=value parameters Should I file an issue for this? Regards, -- Jean Jordaan http://www.upfrontsystems.co.za

2 4

Re: [ZCM] [ZC] 869/ 5 Comment "Broken transaction handling in case of exceptions"
by Toby Dickenson 09 Apr '03

09 Apr '03

Lets take this back to the list.... Ive trimmed the cc list to people I think might be interested. http://collector.zope.org/Zope/869 > I am not in line with your principle "the error report is > part of [the output of] the first transaction". The error report tells you what went wrong. You can only generate an accurate report if you see the objects in the error state. Zope is saying to your application: "your transaction is doomed. tell me why." As you said: > Drawback: you may see newer data in the error report than > has been seen by the erroneous transaction. I think thats a *major* drawback. The risk of zodb badness is a major drawback too..... Small risks add up over time. > But, even with Zope, when the error report should do > something persistently, it must do it in a new transaction > as the old state is inconsistent and can not be committed. I agree that it is unacceptable to commit changes made during the error report without starting a clean new transaction. I agree my proposal is inadequate if we need to allow this. Currently Zope doesnt allow the error report to do something persistently, and the use case for this is not obvious to me. Can you explain the use case? I am hoping that there may be a better solution that avoids the problems I raised. -- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson

4 14