August 2004 - Zope-Dev

Re: Are input streams seekable in Zope? (+ patch proposal)
by Ames Andreas (MPA/DF) 12 Aug '04

12 Aug '04

Hi, at first let me say that I'm very sorry for the long delay; it was just an oversight. Tres Seaver wrote: > Is there a reason why the AJP protocol won't allow you to "rewind" > to the beginning of the request stream? I don't think that the > publisher does any other seek than to the start of the stream. There is nothing in AJP that supports such a rewind but also nothing that prevents it. I think it's very comparable to the HTTP part of the ZServer. The problem probably was that I simply didn't expect 'stdin.seek()' to do something else than throwing an error. I just buffer configurable amounts of bytes of an AJP requests in RAM, no disk buffers or something for avoiding blocking I/O in ZServer. When the configured amount is in memory I just stop reading from the socket, thus I'm relying on TCP's traffic control. The respective ZPublisher thread informs the ZServer, when it has removed sth. from the input buffer, by a trigger event. > Perhaps you need to derive an AJPRequest from HTTPRequest, and > arrange not to need the 'stidn' stream during a retry; another > possibility would be to submit a patch which allowed the retry > mechanism to work without re-parsing the request stream (basically, > the patch would need to clone the cgi.FieldRequest set from the > original request into the one used for the retry). The first solution you propose is the one that immediately came to my mind because it sounds similar to what the ZServer does for HTTP, i.e. buffer big requests on disk smaller ones in StringIO. The cause I don't like that too much is that cgi.FileStorage doesn't let me control if its input is disk buffered once again internally. Writing the same request to disk (and reading it in) potentially twice (or even more often in the retry-case) is a bit too heavy for me. I thought about trying to come up with a patch against cgi.FieldStorage that would leave it to the caller to decide if FieldStorage could do without its internal disk copy and just use the input. But even if I could do that it would take a long time to get effective, so there'd be still the need for an immediate workaround. So your second proposal is what I actually want to try first. I've just finished a patch against ZPublisher.HTTPRequest.py, that I attached to this mail, that, on retry, doesn't read the whole request again from the input stream (as during the first attempt) but just keeps the toplevel cgi.FieldStorage, that has been created by the first attempt in HTTPRequest.processInputs, in a newly created instance variable self._fs. This instance variable is then preserved across retry boundaries. I've tested the patch preliminarily and it worked for me. As I don't know how and where to send a patch against the Zope(2) core (and also because I'd hope for a little review here :-) I'd just be glad if you could tell me how the correct procedure is. I just hope that I wouldn't have to go through some kind of 'paper war' just to get three or four lines of code changed. cheers, andreas

1 0

While Andreas is on vacation....PCNG blues
by Bryan Simmons 12 Aug '04

12 Aug '04

I remember reading somewhere that to change anything in the workflow for plonecollectorng is bad under certain circumstances, such as after you have already created issues. Can anyone elaborate? I'm not finding docs on it through sourceforge or zope.org. -- Regards, Bryan Simmons -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. -- Rick Osborne -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

1 0

KeyError in GlobalTranslationService.py
by Russ Ferriday 11 Aug '04

11 Aug '04

I just submitted a patch for Plone 2.0.3 http://plone.org/collector/3329 that stops a KeyError from being triggered in GlobalTranslationService.py when a msgid contains a "$" I think that passing a translation key containing $ should not cause an Error page. So here's a patch for Zope that at least prevents the problem. Translation Service dudes will probably pick big holes in it. Oh, and traceback attached below. The msgid was Ca$hville, the title of a doc. --r. diff GlobalTranslationService.py.orig GlobalTranslationService.py 28,31c28,35 < def repl(m, mapping=mapping): < return ustr(mapping[m.group(m.lastindex)]) < cre = re.compile(r'\$(?:(%s)|\{(%s)\})' % (NAME_RE, NAME_RE)) < return cre.sub(repl, default or msgid) --- > try: > def repl(m, mapping=mapping): > return ustr(mapping[m.group(m.lastindex)]) > cre = re.compile(r'\$(?:(%s)|\{(%s)\})' % (NAME_RE, NAME_RE)) > return cre.sub(repl, default or msgid) > except KeyError: > return msgid Traceback: Site Error An error was encountered while publishing this resource. KeyError Sorry, a site error occurred. Traceback (innermost last): * Module ZPublisher.Publish, line 163, in publish_module_standard * Module ZPublisher.Publish, line 127, in publish * Module Zope.App.startup, line 203, in zpublisher_exception_hook * Module ZPublisher.Publish, line 100, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 40, in call_object * Module Shared.DC.Scripts.Bindings, line 306, in __call__ * Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec * Module Products.CMFCore.FSPageTemplate, line 191, in _exec * Module Products.CMFCore.FSPageTemplate, line 124, in pt_render * Module Products.PageTemplates.PageTemplate, line 96, in pt_render <FSPageTemplate at /testdollarproblem/document_view used for /testdollarproblem/foo> * Module TAL.TALInterpreter, line 189, in __call__ * Module TAL.TALInterpreter, line 233, in interpret * Module TAL.TALInterpreter, line 663, in do_useMacro * Module TAL.TALInterpreter, line 233, in interpret * Module TAL.TALInterpreter, line 408, in do_optTag_tal * Module TAL.TALInterpreter, line 393, in do_optTag * Module TAL.TALInterpreter, line 388, in no_tag * Module TAL.TALInterpreter, line 233, in interpret * Module TAL.TALInterpreter, line 663, in do_useMacro * Module TAL.TALInterpreter, line 233, in interpret * Module TAL.TALInterpreter, line 605, in do_loop_tal * Module TAL.TALInterpreter, line 233, in interpret * Module TAL.TALInterpreter, line 408, in do_optTag_tal * Module TAL.TALInterpreter, line 393, in do_optTag * Module TAL.TALInterpreter, line 388, in no_tag * Module TAL.TALInterpreter, line 233, in interpret * Module TAL.TALInterpreter, line 629, in do_condition * Module TAL.TALInterpreter, line 233, in interpret * Module TAL.TALInterpreter, line 552, in do_insertTranslation * Module TAL.TALInterpreter, line 615, in translate * Module Products.PageTemplates.TALES, line 263, in translate * Module Products.PageTemplates.GlobalTranslationService, line 31, in translate * Module Products.PageTemplates.GlobalTranslationService, line 29, in repl KeyError: 'hville' (Also, an error occurred while attempting to render the standard error message.) ————————————————————— Russ Ferriday Solution Workshops for Plone (+44) (0) 7789 338868 http://www.solutionworkshops.com

2 1

Collector Status Meanings
by Chris Withers 10 Aug '04

10 Aug '04

Hi All, Apologies for the cross-posting, but I think this is relevent to all these lists. I've summarised the meaning of the various collector states here: http://dev.zope.org/CVS/CollectorStatuses Please let me know if you disagre with any of that, although I'm pretty sure they're right and will argue with anyone who thinks otherwise ;-) The only real change is that Deferred now means "we asked the user for more information and we'll reject the issue unless they give it to us within a month" I went through all the issues which WERE deferred and "dealt" with them. I'm trying to avoid having states where issues end up that aren't definitive and so get forgotten about. The "wontfix" stuff now has a definitve meaning, but it may still be good to go through them all once a year or so to check that none of them have been solved in other ways. I found quite a few of the "deferred" ones that really should have been "wontfix" had been addressed and could now be marked as resolved :-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

8 9

Problems with session on ZEO
by Gerhard Schmidt 09 Aug '04

09 Aug '04

Hi, we have some problems with our tempstorage on out ZEO client. We are running 6 zope Frontend server and one ZEO server. Serving Both main and temp storage. I runs well for some days. But than it stops working an reports the following error Traceback (innermost last): Module ZPublisher.Publish, line 101, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 39, in call_object Module Shared.DC.Scripts.Bindings, line 306, in __call__ Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec Module Products.PageTemplates.ZopePageTemplate, line 222, in _exec Module Products.PageTemplates.PageTemplate, line 96, in pt_render - <ZopePageTemplate at /TUM_Portal/login_form> Module TAL.TALInterpreter, line 189, in __call__ Module TAL.TALInterpreter, line 218, in interpret Module TAL.TALInterpreter, line 233, in interpret Module TAL.TALInterpreter, line 670, in do_useMacro Module TAL.TALInterpreter, line 233, in interpret Module TAL.TALInterpreter, line 693, in do_defineSlot Module TAL.TALInterpreter, line 233, in interpret Module TAL.TALInterpreter, line 636, in do_condition Module TAL.TALInterpreter, line 233, in interpret Module TAL.TALInterpreter, line 483, in do_insertText_tal Module Products.PageTemplates.TALES, line 227, in evaluateText Module Products.PageTemplates.TALES, line 221, in evaluate Module Products.PageTemplates.ZRPythonExpr, line 47, in __call__ - __traceback_info__: here.REQUEST.SESSION.set('callerURL', here.REQUEST.HTTP_REFERER) Module Python expression "here.REQUEST.SESSION.set('callerURL', here.REQUEST.HTTP_REFERER)", line 1, in <expression> Module ZPublisher.HTTPRequest, line 1218, in __getattr__ Module ZPublisher.HTTPRequest, line 1178, in get Module Products.Sessions.SessionDataManager, line 94, in getSessionData Module Products.Sessions.SessionDataManager, line 183, in _getSessionDataObject SystemError: error return without exception set Restarting the zeo server and one frontend fixes the problem for some time. i am running Zope 2.7.2 and Python 2.3.4 Here is the zeo config %define INSTANCE_HOME /data/zope/zeo <zeo> address 7700 read-only false invalidation-queue-size 500 # monitor-address PORT transaction-timeout 10 </zeo> <filestorage main> path $INSTANCE_HOME/var/Data.fs </filestorage> %import tempstorage <temporarystorage temp> name temporary storage for sessioning </temporarystorage> <eventlog> level info <logfile> path $INSTANCE_HOME/log/zeo.log </logfile> </eventlog> Bye Estartu ---------------------------------------------------------------------------- Gerhard Schmidt | Nick : estartu IRC : Estartu | PGP Public Key Fischbachweg 3 | Privat: estartu(a)augusta.de | auf Anfrage/ 86856 Hiltenfingen | Dienst: schmidt(a)ze.tu-muenchen.de | on request

5 8

Making a ZSQL.DA fully multi-threaded?
by Brad Clements 08 Aug '04

08 Aug '04

I am using ZsapdbDA with SAPDB (now MaxDB) and have run into problems when Zope has more than one thread. The folks at SAP suggest that there should be one database connection per thread, rather than sharing one connection across threads. I haven't yet looked at the Zsapdb source, but I think it has only one connection for all threads. Is there a document somewhere that explains how to make variables "thread private" in Zope? I vaguely recall something about one cache per thread, or that variables are read- only shared unless updated by a thread, and then they're copied.. This is probably quite wrong, but obviously I just don't understand how data and threading works in Zope. So, I'm looking for a document that explains this clearly. How to have data that is not shared between threads (perhaps a kind of connection pool thingy). Also, I have other projects where I really want to have just one object of it's kind no matter how many threads. For example, gvibDA uses Thunked_TM for this, but that doesn't really work with ZSQLDA because I've seen that multiple connect calls are made, even with the THUNKED_TM. Is there a "Zen Of" document about this somewhere? Thanks -- Brad Clements, bkc(a)murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements

7 26

Re: [Zope-dev] Syslog logger broken?
by Paul Winkler 05 Aug '04

05 Aug '04

On Sun, Aug 01, 2004 at 06:20:19AM +0200, Andreas Jung wrote: > > > --On Samstag, 31. Juli 2004 23:37 Uhr -0400 Paul Winkler > <pw_lists(a)slinkp.com> wrote: > > >Has anybody had success with the syslog logger in Zope 2.7.0 or later? > >AFAICT the example in zope.conf.in is borken. > >See http://collector.zope.org/Zope/1425 > > > >Andreas assigned it to me, but I really have no clue > >how to fix it. > > > > > > hm...according to your report only the entries in the example are > broken...maybe I have to > re-read again.. Yes, please do :-) >From the bug report: """ The following is what I guess to be correct based on zLOG/component.xml, and it raises no errors, but I don't see any output either (maybe it's a problem with my /etc/syslog.conf): <logger trace> level all <syslog> address localhost:514 </syslog> </logger> """ -- Paul Winkler http://www.slinkp.com

1 1

ZConfig keys
by Godefroid Chapelle 04 Aug '04

04 Aug '04

Hello, I would like to confirm that ZConfig keys are case insensitive and that the corresponding attributes on the config object returned by the 'loadConfig' call are always lower case. The only thing that seems relevant about keys and case sensitivity in the documentation does not state it explicitely : '''When this schema is loaded, a set of defaults for the derived section type is computed. Since basic-key is case-insensitive (everything is converted to lower case), ‘foo’ and ‘Foo’ are both converted to ‘foo’, which clashes since key only allows one value for each key.''' Thanks -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be

2 1

More POSKeyErrors than I know what to do with
by Richard Jones 04 Aug '04

04 Aug '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Our 2.7.1-based production server has started throwing up a lot of POSKeyErrors recently. Forcing regeneration of the Data.fs.index didn't fix it. Packing didn't fix it or generate any new errors - we haven't packed the database since it went live. fsrecover.py doesn't do anything. Running fsrefs.py on the Data.fs gives *scary* long output (attached both unpacked and packed ouput). I haven't been able to get checkbtrees.py to produce any useful output (it doesn't appear to want to poke into the Application). I'm not sure it'll tell me any more than fsrefs.py though. - From my reading of the fsrefs.py output, it looks like the corruption is the result of a single transaction. I'm more than happy to look into this further, and also to supply the Data.fs (200Mb unpacked, 20Mb packed) to help analyse it. Richard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBEDKUrGisBEHG6TARAo4cAJ9xuXtYf95uJf/oj+95CfdFJ9qNTgCfaRT7 xETqjHTNProIG6n1Yr1vjp8= =moPK -----END PGP SIGNATURE-----

1 0

[Fwd: Re: [Zope-dev] Referencial Integrity in ZCatalogs]
by Leonardo Rochael Almeida 03 Aug '04

03 Aug '04

Forgot to Cc: zope-dev -----Mensagem encaminhada----- From: Leonardo Rochael Almeida <leo(a)hiper.com.br> To: Santi Camps <scamps(a)earcon.com> Subject: Re: [Zope-dev] Referencial Integrity in ZCatalogs Date: Mon, 02 Aug 2004 17:05:53 -0300 I would suggest you take a look at mxmRelations, however Zope.org workflow is keeping it out of reach, and I couldn't find it on MaxM's own site. The URL for it would be: http://www.zope.org/Members/maxm/products/mxmRelations but right now it requires a login. If any kind soul from the Zope web team could release it, or if MaxM could post it somewhere else, that would be very nice. Cheers, Leo

3 2