- Zope-Dev - Zope lists

ZCatalog peculiarities?
by Phillip J. Eby 16 Aug '99

16 Aug '99

I've been spelunking through the ZCatalog source this morning, and there are a couple of things I don't understand, that look to me like memory leaks and other problems. First, in Catalog.py, it seems like virtually every method of the Catalog class calls self.useBrains(). useBrains() creates a new class each time it's called, and then deliberately creates a circular reference in that class, such that it can't be deleted. Finally, the logic of where useBrains() is called, is such that it will be called once for every item retrieved from every search done on the catalog. Thus, every record should end up as an instance of a different class, specifically created for that record. This has me baffled, because I can't see why that would be something useful. Am I missing something really deep here, or is it just a bug? It would seem to me that there should be something like "_v_brains_inuse =_v_brains" at the end of the useBrains() method, so that at the beginning you can check for whether "brains is _v_brains_inuse" and skip creating a new class, etc. once the brains are initialized. The next thing I've noticed, is that Catalog.py imports from Query, but then never actually uses anything from it. Oh, sure, there's an orify() function that does stuff, and a query_map local variable set up in searchResults() that references orify, but query_map never gets used. (And ZCatalog.py makes up another copy of the same query_map object and passes it in, but that never gets used either.) I even dug down to Index.py and TextIndex.py to make sure query_map wasn't used down at that level, and then I realized that query_map wasn't part of the namespace being passed down there anyway. This all looks like some sort of vestigial leftovers, perhaps during porting from ZTables...? The third thing I saw, is that there are lots of constructs of the form: if something in somethingelse.keys(): where the object 'somethingelse' appears to have a perfectly good has_key() method available. Again, is there some sort of deep Zen here, like trying to force index pages to be loaded, or...? I mean, it seems like this would be much less efficient than just hitting the has_key() methods. Last, and definitely least, I notice that useBrains() uses a loop to create scopy, when "scopy=self.schema.copy()" would suffice. That wouldn't be anything worth 'fixing', except considering how often that code would appear to be being called. I'm really tempted to run something through the debugger or profiler just to see if I'm totally off base on how often useBrains() is getting called. But I thought I'd run this by the experts, since if y'all did this stuff on purpose, you'd be able to tell me. :) Oh, one other thing. Not a bug, but a feature... It seems to me that if you are searching a ZCatalog on index 'foo', it looks possible to do something like: Catalog(foo=['bar','baz'],foo_usage='range:min:max') And receive records with values of foo ranging from 'bar' to 'baz'. Is this a supported, intended feature, or...? Just wondering.

2 1

Re: [Zope] Alpha of ODP Viewer product
by Michel Pelletier 16 Aug '99

16 Aug '99

"Andrew M. Kuchling" wrote: > > Maik Roeder writes: > >What do you mean by "get the *real* XML dumps into Zope" ? > > Various pieces don't expect to handle 45Mb of input data. For > example, ZPublisher.Client handles file uploads by reading all the > data into memory, and it dies; pcgi-wrapper tries to read all of stdin > and also dies, etc. I'm trying to do it from the monitor_client with > the following incantation: > > import Zope ; a=Zope.app() > a.manage_addProduct['XMLDocument'].add(a, > id='s1', title='Structure', > file=open('structure.rdf', 'r') ) > > I haven't made this work yet, and suspect this is also choking > somewhere along the way. Anyone from Digicool ever tried to shovel > this much data into an XMLDocument? > Sub transactions can help in this particular case, I believe. A quick glance at the XMLDocument code looks like it doesn't use subtransactions like File objects do (see lib/python/OFS/Image.py manage_addFile method). I could be wrong, but it looks like a pretty simple change. Amos? -Michel > -- > A.M. Kuchling http://starship.python.net/crew/amk/ > A slovenly action repeated thrice has become a habit. > -- Robertson Davies, _Leaven of Malice_ > > _______________________________________________ > Zope maillist - Zope(a)zope.org > http://www.zope.org/mailman/listinfo/zope > > (To receive general Zope announcements, see: > http://www.zope.org/mailman/listinfo/zope-announce > > For developer-specific issues, zope-dev(a)zope.org - > http://www.zope.org/mailman/listinfo/zope-dev )

1 0

Calendar TAG Error : Unrecognized timezone
by kentsin 14 Aug '99

14 Aug '99

I encount the following error when using calendar tag : Error Type: DateTimeError Error Value: Unrecognized timezone The code is  <h1>Testing</h1>     The system is in win98 with time zone set to GMT+08:00 Beijing, Chongqing, Hong Kong, Urumqi

1 0

Re: [Zope-dev] Trying to design a PythonMethod Product
by Christopher Petrilli 13 Aug '99

13 Aug '99

>> Look at the expression machinery, it would be insane to duplicate security >> already implemented, and frustrating to have two separate "paradigms" for >> managing security. > > Surely! This consists of DT_Util, VSEval, and gparser, yes? As far as I > can tell, these perform two tasks; They screen out restricted names and > guard selected operations. You folks may want to seriously consider > replacing gparser with something using bytecodehacks; Walking an AST is > terribly opaque and complex compared to manipulating an EditableCode object. > That's how I'm doing things, anyway, and I'll do my best to use the > information and guard functions in DT_Util directly. Hmm. For that matter, > I may try to generalize enough to offer a plug-in replacement for VSEval. I would be curious of the performance difference as well. We wrote our stuff LONG before anyone had developed anything better, and it was heavily optimized. Even a 5-10% performance hit is probably unacceptable in DTML, given it needs to be faster as it is, and adding additional overhead is silly. It would have to benefit Zope at some level above just making it more "obvious" how an AST is walked... 99.999999999% of users will never see it, nor should they want to. >> Actually, we talked about just limiting the "instruction counter" so that >> you could catch anything that was taking an excessively long period of > time >> to execute. Say 1M byte-codes? This avoids having to worry about >> individual flow constructs. > > Sounds great! How? :-) I'll have to look, but I recall Jim explaining a way to do this, so I'm sure it's possible. >> > 7. Either disallow "global", or redirect access to globals into a > controlled >> > namespace. >> >> Look at management of namespaces in expressions. > > This part is proving to be fairly deep. DTML objects require a client and > mapping to operate on. I want PythonMethods to be a*lot* like External > Methods, with any number of parameters (including zero) and the option to > pass '_' or 'REQUEST' explicitly. Under this regime, global names can't > refer to Zope namespace contents; So far, I've got them accessing a volatile > private namespace. Hehehe, namespaces are *very* deep indeed, and are one of the difficulties in doing things that people think should be "easy". Our goal was NOT to make "External Methods" internal, but rather to allow more complex Python expression web enabled. The former is a very dangerous path to walk down from a security perspective. In fact, in our "image" it wouldn't even have a "def" statement, nor could you define classes, etc., it was simply for encapsulating more complex logic. > This has led me to extend my original idea. I *really* dislike requiring a > separate Zope object for every method in a module of external methods, and I > think it would be even more annoying when the actual code is web-managed. > Now I'm thinking of creating a PythonModule product, which could define and > expose any number of functions and classes. These would act like methods of > the PythonModule. Perhaps defining an object named 'default' (or > 'index_html'? <bleah>) would allow the PM to be called/rendered directly. > Now, all the objects in the PM would have a meaningful shared global > namespace, just as if they were in a regular Python module. I disagree strongly here. An "method" is an object, and it might reside in a container (which one might call a Module), but modules are namespaces, not objects in the sense that you're using them. I think your goals, while admirable, are substantially large enough in scope to mire down the implementation with additional chrome that may or may not be useful. I'd recommend getting something very basic done before attempting the Holy Grail. I very much think you should use the existing machinery, rather than designing your own new... it's just more variables to examine when trying to understand the isolation mechanisms. Chris -- | Christopher Petrilli Python Powered Digital Creations, Inc. | petrilli(a)digicool.com http://www.digicool.com

2 1

exception under load
by Robin Becker 13 Aug '99

13 Aug '99

I see this now and then with loaded CVS Zope 1999-08-13T16:56:52 ERROR(200) ZServer uncaptured python exception, closing chan nel <zhttp_channel connected 127.0.0.1:1680 at 1ee89a0 channel#: 854 requests:1> (exceptions.AttributeError:'int' object has no attribute 'send' [C:\Python\deve l\Zope\ZServer\medusa\asyncore.py|poll|64] [C:\Python\devel\Zope\ZServer\medusa\ asyncore.py|handle_write_event|298] [C:\Python\devel\ZOPE\ZServer\medusa\asyncha t.py|handle_write|144] [C:\Python\devel\ZOPE\ZServer\medusa\asynchat.py|initiate _send|211] [C:\Python\devel\Zope\ZServer\medusa\http_server.py|send|394] [C:\Pyt hon\devel\Zope\ZServer\medusa\asyncore.py|send|231] [<string>|send|1]) -- Robin Becker

1 0

Re: [Zope-dev] Zope scheduler any one.
by Josh Zeidner 13 Aug '99

13 Aug '99

You could write a simple system script to contact a URL inside sope fairly easily. -Josh Z >From: Robin Becker <robin(a)jessikat.demon.co.uk> >To: zope-dev(a)zope.org >Subject: [Zope-dev] Zope scheduler any one. >Date: Fri, 13 Aug 1999 10:46:27 +0100 > >Any plans for a Zope scheduler ie a means of getting certain things done >periodically. I would like to pack the database automatically fairly >regularly, and periodically remove certain user folders and the like. >-- >Robin Becker > >_______________________________________________ >Zope-Dev maillist - Zope-Dev(a)zope.org >http://www.zope.org/mailman/listinfo/zope-dev > >(To receive general Zope announcements, see: >http://www.zope.org/mailman/listinfo/zope-announce > >For non-developer, user-level issues, >zope(a)zope.org, http://www.zope.org/mailman/listinfo/zope ) _______________________________________________________________ Get Free Email and Do More On The Web. Visit http://www.msn.com

1 0

Re: [Zope-dev] Trying to design a PythonMethod Product
by Evan Simpson 13 Aug '99

13 Aug '99

From: Christopher Petrilli <petrilli(a)digicool.com> > I'll make some comments since we've tossed this around ENORMOUSLY in house > and just haven't had the time to write it. It's really not that hard I > think, but you do need to take into account some issues. I have working code going already, but I'm cleaning it up for general consumption. Any help getting the issues right is greatly appreciated. > Look at the expression machinery, it would be insane to duplicate security > already implemented, and frustrating to have two separate "paradigms" for > managing security. Surely! This consists of DT_Util, VSEval, and gparser, yes? As far as I can tell, these perform two tasks; They screen out restricted names and guard selected operations. You folks may want to seriously consider replacing gparser with something using bytecodehacks; Walking an AST is terribly opaque and complex compared to manipulating an EditableCode object. That's how I'm doing things, anyway, and I'll do my best to use the information and guard functions in DT_Util directly. Hmm. For that matter, I may try to generalize enough to offer a plug-in replacement for VSEval. > > 4. Restrict "import" to modules/packages found in a particular location, so > > that 'safe' modules can be used. > > Take a look at rexec for this. I have, and may use some bits of it. Should we allow import of standard modules, or even Extensions modules? I'm leery of that, and lean toward requiring explicitly exposing selected modules to import. Heck, I may even have 'import' bring in names from the Zope object hierarchy instead of external files! > Actually, we talked about just limiting the "instruction counter" so that > you could catch anything that was taking an excessively long period of time > to execute. Say 1M byte-codes? This avoids having to worry about > individual flow constructs. Sounds great! How? :-) > Not sure how I like this honestly. I'd probably just bind sys.stdout.write > to something new, that should cause print to work correctly. I've got this part working to my satisfaction already. > > 7. Either disallow "global", or redirect access to globals into a controlled > > namespace. > > Look at management of namespaces in expressions. This part is proving to be fairly deep. DTML objects require a client and mapping to operate on. I want PythonMethods to be a*lot* like External Methods, with any number of parameters (including zero) and the option to pass '_' or 'REQUEST' explicitly. Under this regime, global names can't refer to Zope namespace contents; So far, I've got them accessing a volatile private namespace. This has led me to extend my original idea. I *really* dislike requiring a separate Zope object for every method in a module of external methods, and I think it would be even more annoying when the actual code is web-managed. Now I'm thinking of creating a PythonModule product, which could define and expose any number of functions and classes. These would act like methods of the PythonModule. Perhaps defining an object named 'default' (or 'index_html'? <bleah>) would allow the PM to be called/rendered directly. Now, all the objects in the PM would have a meaningful shared global namespace, just as if they were in a regular Python module. So far, I've got restricted assignment, print interception (and retrieval), 'del' and 'exec' blocking, a safe global namespace, and I'm comfortable with mucking about in bytecodes. Import handling and operator guards should be straighforward (once I know what import should *do* <wink>). Keep those ideas rolling, Evan Simpson

1 0

Re: [Zope-dev] objectValues(spec) does not work!?
by Jonas Juselius 13 Aug '99

13 Aug '99

Does not work, and does not matter... If you look at the code in ObjectManager, objectItems() calls objectIds() with 'spec', and objectIds() tests whether 'spec' is a string or not, and if it is a string it converts it into a list. Sorry about the typo in my previous post, my code _actually_ read objectItems(['Folder']) not objectItems('[Folder]'). jonas On Fri, Aug 13, 1999 at 11:14:28AM -0400, Christopher Petrilli wrote: > > I have the following problem in dtml: > > > >  > > . > > . > > . > >  > > > > does not work! Neither does objectItems('[Folder]'), but a plain > >  works... I have looked at the code in ObjectManager.py > > and DT_Util.py, but can't figure it out... > > You're mispelling it: > >  > > objectItems, along with other such things expect a *list* of meta-types. > > Chris > > -- > | Christopher Petrilli Python Powered Digital Creations, Inc. > | petrilli(a)digicool.com http://www.digicool.com -- And what is good, Phaedrus, And what is not--- Need we ask anyone to tell us these things? [ PGP public key: finger jonas(a)radon.pc.helsinki.fi ]

1 0

quibble about directory structure
by Joel Berendzen 13 Aug '99

13 Aug '99

Dear Zope Developers, I'm manager of a development project for a scientific network-based compute appliance that will be Zope-based. I have only limited experience with Zope (starting with 2.0.0a2) but I'm an experienced programmer and sysadmin. I'm getting to be experienced with installing new Zope releases, which is where my quibble comes from. My quibble is that doing updates to Zope is harder than it should be because of the directory structure. Life would be sweet in Zopeville if everything that varies from installation to installation would be kept in var. This way, I could simply keep a var directory in a place outside the Zope tree itself and make a symlink to it as part of the install. Instead, I have var, Extensions, import (I think, haven't used this feature yet), and (the worst) parts of lib/python/Products that have to be moved with each install. I would like to suggest that the Extensions and import directory be moved to var in future releases, that var/Products be created and put on the search path, and that the rest of the tree be specified as read-only. I can of course do this myself with symlinks (and I will), I think a change in the official distribution might save us all a little trouble. Cheers, Joel -- Joel Berendzen Biophysics Group (P-21), D454 Phone: +1.505.665.2552 Los Alamos National Laboratory FAX: +1.505.665.4507 Los Alamos, NM 87545, USA. email: joelb(a)lanl.gov

1 0

Re: [Zope-dev] objectValues(spec) does not work!?
by Christopher Petrilli 13 Aug '99

13 Aug '99

> I have the following problem in dtml: > >  > . > . > . >  > > does not work! Neither does objectItems('[Folder]'), but a plain >  works... I have looked at the code in ObjectManager.py > and DT_Util.py, but can't figure it out... You're mispelling it:  objectItems, along with other such things expect a *list* of meta-types. Chris -- | Christopher Petrilli Python Powered Digital Creations, Inc. | petrilli(a)digicool.com http://www.digicool.com

1 0