November 2002 - Zope-Dev

Re: [Zope] Cache Parameters
by Brian R Brinegar 25 Nov '02

25 Nov '02

Is there a way in Zope 2.5 to see what kind of hit/miss ratio our cache is having? Currently we have 260,000 objects in our database and an ideal cache size of 10,000. And I'm not sure how often it's accessing the database. Thanks, -Brian On Fri, 22 Nov 2002, Toby Dickenson wrote: > On Friday 22 November 2002 10:18 am, Chris Withers wrote: > > Brian R Brinegar wrote: > > > Within the Control_Panel/Database/ Cache Parameters there is a "Target > > > max time between accesses" what is the Unit for this value? Seconds? > > > Minutes? I haven't found it documented anywhere. > > > > Best ask Toby Dickenson on the zodb-dev(a)zope.org list, he is the god of all > > things cache like ;-) > > No need to go to a specific list. I am everywhere. ;-) > > > That parameter has been removed in Zope 2.6. Previously its units were in > seconds, but the parameter that it controls does not relate to anything > easily explained. > > If you think you need to understand this parameter, then I think you need to > upgrade to Zope 2.6 > > >

2 1

PathIndex unindex fix
by seb bacon 24 Nov '02

24 Nov '02

Anyone object to me changing PathIndex so it swallows exceptions and logs them when unindexing content? On upgrading to Zope 2.6 some if the unindex paths seem to have got hosed, and I don't want this breaking my site.

6 8

adding properties to instances v. classes
by Jamie Heilman 23 Nov '02

23 Nov '02

There's this somewhat annoying issues surrounding objects that declare removable properties in their classes, in that if you remove them, you can't add them again because many times a class variable of the same name already exists. (atleast thats my understanding of it, I suppose I could be smoking something) The issue was raised in bug 337 which was subsequently closed after the Image class got a slight massaging. I'm wondering if there's any reason why simply adding the properties and instance variables to the instance within __init__ would be a bad thing for properties that you want to declare removable. For example: class Baz(Stuff): """meh.""" title = '' _properties = ({'id':'title', 'type': 'string', 'mode':'w'},) def __init__(self, id, title, foo): self._properties += ({'id':'foo', 'type':'string'},) self.foo = foo self.title = title ... Seems like it would be a fairly easy fix and it certainly seems to work, am I missing something? I'm also wondering what the deal is with the title property, should it alway be removable, always be unremovable, or always be what you want it to be depending on your mood ring? It doesn't seem like there's a great deal of consistency and I'm wondering if thats intentional. -- Jamie Heilman http://audible.transient.net/~jamie/ "It's almost impossible to overestimate the unimportance of most things." -John Logue

2 3

Fw: Rapid Alternated Propositions
by fr 23 Nov '02

23 Nov '02

Hi All, 1/ In ZMI : it could be cool, in general, to select all the checkboxes in one time with a little javascript function or something else ... 2/ In Security ZMI item : Why not group permissions by product or group in a general category when permissions are commons FR

3 2

Interfaces, HelpSys and API documentation
by Yuppie 23 Nov '02

23 Nov '02

Hi! I'm currently digging into Zope coding conventions. And I can't find any guidance how to use Zope Interfaces for API documentation in Zope's online help system. I found these related pages, but they are all out of date: <http://www.zope.org/Wikis/Interfaces/> <http://www.zope.org/Members/michel/Projects/Interfaces/> <http://dev.zope.org/Wikis/DevSite/Projects/APIDocs/> <http://www.zope.org/DocProjects/zope_product> The ZDG doesn't reflect the current API and mentions API help with just one word: <http://www.zope.org/Documentation/Books/ZDG/current/ComponentsAndInterfaces…> <http://www.zope.org/Documentation/Books/ZDG/current/Products.stx> I think it is a Good Thing to use interfaces to generate API help, because - using Interface.Verify you have a basic test if API documentation and implementation are in sync - you don't have to write API documentation and Interfaces But API Reference syntax and Interface syntax are not identical. So far only some CMFCore interfaces follow this philosophy. (If you want to see a working example, look at WorkflowTool.py, interfaces\portal_workflow.py and tests\test_WorkflowTool.py) But this example also shows the problem: Reading the online API help, most people are interested in APIs they can use, not interface definitions. They are looking for 'portal_workflow', not 'WorkflowTool' or 'IWorkflowTool' or 'WorkflowToolInterface'. In this example it is "solved" by using the same id for the interface and the unique object. The docstrings sometimes refer to "this tool" and sometimes to the "tool interface". Did I miss some useful docs? Any ideas how to resolve this? TIA, Yuppie

1 0

Rapid Alternated Propositions
by fr 22 Nov '02

22 Nov '02

Hi All, 1/ In ZMI : it could be cool, in general, to select all the checkboxes in one time with a little javascript function or something else ... 2/ if it is possible, why not group by product permissions, else in a general category if permissions are commons FR

1 0

Zope Caches
by Brian R Brinegar 22 Nov '02

22 Nov '02

Okay, I'm trying to better understand Caching within Zope. Is a cache created for each Zope Thread? Or one per Zope instance? We have 3 ZEO Clients with 4 Threads each. I want to know if we have 3 copies of the cache, or 12 copies of the cache. Thanks, -Brian

4 13

Re: Unsecure design of ExternalFile
by Craeg K Strong 22 Nov '02

22 Nov '02

Yikes! Scary stuff. However, here are some things to consider: a) ExternalFile advertises itself as being a developers/ content authors tool, not really for production. Of course, most folks end up using it for production, anyway... ;-) b) Once created, an ExternalFile cannot be retargetted to point to another file in the file system c) The permission to create an ExternalFile instance is different than the permission to edit one. The permission to create an ExternalFile instance should be assigned judiciously... d) the Zope server should be run as a user that has very limited permissions. e) Even if a user *does* have permission to edit an ExternalFile, they only have whatever permission the user running the Zope server has. If the Zope user (usually "webserver" or something like that) does not have permission to write to /etc/passwd, it doesn't matter if you create an ExternalFile pointing to it, you still can't write to it... However, the points you raise are valid, as they are Zope-specific, and the zope user "webserver" *would* probably have permission to do your (1) and (2) examples. What would you recommend? Perhaps there should be a predefined list of "forbidden" directories for ExternalFiles? The problem is that-- in the development scenario-- the very things you mention below might be what you legitimately *want* to do as a developer. Well, thanks for pointing this out. Let's continue the conversation a bit, perhaps a good solution will reveal itself (even if it is only some kind of warning in the documentation...) Regards, --Craeg PS I am CC-ing the zope-dev mailing list, as I think this warrants a wider audience Jonagustine Lim wrote: > Hi! > > I just noticed that it's possible to create or replace > any files in the filesystem using the ZMI with > ExternalFile installed. > > Possible exploits: > > 1. Use ExternalFile web interface to replace Zope > Data.fs > > 2. Create a .py file in /Zope/Extensions and run it by > creating an Extenal Method. > > Anyway, I hope you can fix this or put a warning up. > > Jon >

5 5

Single-threaded Zope
by Carlo Giomini 22 Nov '02

22 Nov '02

Dear all, because of reasons that would be boring to explain,I have the following requirement for my site: I need Zope NOT to run as a single-process multithreaded application (as it does by default) but instead to run as a pool of processes (each one single-threaded); those processes should share the same Data.fs database. Shutting down the threads is trivial (using setNumberOfThreads function) but the tricky part (for me, at least) is to keep the ability to serve many requests concurrently. Since I discarded threads, the only other way is to have many Zope instances running (as independent single-threaded processes) and a sort of dispatcher process (proxy server?) that listens for incoming HTTP connections from the outside world and dispatches it to one of the Zope instances (performing also the reverse operation, of course). The dispatcher should also care to create (and destroy) dynamically the Zope instances, according to the load of traffic (number of requests per second). Is there any way to do so without having to implement the dispatcher (proxy) process from scratch? I have been considering FastCGI as a solution, since it allows a web server (say Apache) to spawn many persistent processes to execute the CGI requests it receives but, to my disappointment, I noticed that the mod_fastcgi directive used in all the howtos on the subject is FastCgiExternalServer, which does not allow for the CGI app to be created by Apache itself. FastCgiExternalServer relies on a 'manual' external starting of the CGI app, which can only be ONE process then! There's another directive (FastCgiConfig) that lets Apache spawn the process(es) that execute the CGI app, the question is: can it be used with Zope instead of FastCgiExternalServer? Since (Fast)CGI apps use pipes to communicate with the web server (when they run on the same machine), the problem becomes how to pass a web request directly to the Zope publishing mechanism over a pipe, bypassing the Zope's HTTP server. How can I do that? Someone out there has ever tried to do so? Also, how does Zope support having multiple instances of it sharing the same database (Data.fs)? One solution could be having many ZEO clients on the same machine, but I wonder if it is possible to create those clients dynamically (to scale according to the load of incoming requests) or rather they can only be installed 'statically' (not at run-time, I mean). Thanks in advance for any help you can give, regards, Carlo.

2 1

Dynamically altering Product methods
by boa13＠free.fr 21 Nov '02

21 Nov '02

Hello, I've been playing a bit with the current implementation of WebService that is availalble from Zope.org. I have tried to adapt the ServiceProxy to a Zope Product, so that a user can simply drop such a WebServiceProxy in a Folder, type-in the URL of a WSDL file and then call the methods described in the WSDL on the WebServiceProxy. A first prototype incorporated a ServiceProxy instance and offered hard-coded methods: def __init__(self, id): self.id = id self.gsp = ServiceProxy("http://api.google.com/GoogleSearch.wsdl") security.declarePublic('doSpellingSuggestion') def doSpellingSuggestion(self, key, phrase): "Checks the phrase for correct spelling" return self.gsp.doSpellingSuggestion(key, phrase) It works well, except for a persistence problem that got me stuck for a while. The WebService package makes use of weakrefs, and I guess this caused problems. However, removing the usage of weakrefs in ServiceProxy alone seemed to solve the problem. (If someone can explain that to me, I'm listening!) While this small test worked well, I wanted to do better: notably, I want the user to be able to change the WSDL on the fly, without needing to destroy the object and to add a new one, and also, I want the user to be able to alter the permission settings for each method in the Web Service. Well, it seems my wishes have made me enter quite a strange realm of low-level Zope mechanisms, which I don't always fully grasp! My current implementation suffers from heavy persistence problems, and contains the following significant code: def setWSDL(self, wsdl, service=None, port=None): """ Sets the Web Service this object proxies. This resets all the Web Service methods information, notably the security settings. """ # Remove old WSDL data if hasattr(self, '_serializer'): binding = self._port.getBinding() portType = binding.getPortType() for item in portType.operations: delattr(self, item.name) delattr(self, item.name+'__roles__') del self._wsdl del self._name del self._port del self._service del self._transport del self._serializer # Add new WSDL data self._security = ClassSecurityInfo() declareStaticPermissions(self._security) # This code mostly comes from WebService.ServiceProxy if not hasattr(wsdl, 'targetNamespace'): wsdl = WSDLReader().loadFromURL(wsdl) self._serializer = Serializer() self._transport = HTTPTransport() for item in wsdl.types.items(): self._serializer.loadSchema(item) self._service = wsdl.services[service or 0] self._port = self._service.ports[port or 0] self._name = self._service.name self._wsdl = wsdl binding = self._port.getBinding() portType = binding.getPortType() for item in portType.operations: callinfo = callInfoFromWSDL(self._port, item.name) method = MethodProxy(self, callinfo) setattr(self, item.name, method) # Default security for the Web Service methods self._security.declareProtected('Use Web Service', item.name) self._security.apply(self) InitializeClass(self) declareStaticPermissions() is a module-level function that declares the security settings for my so-called "static" methods, such as setWSDL, which are intended to always be present in the WebServiceProxy. Now, when I use this product, it works well the first time. I can set the WSDL to the Google Web Service, and then call its spellchecking method successfully. However, as soon as persistence comes into play, I'm toast. Simply switching directories a few times is enough for that. I then get the following error: 2002-11-20T12:20:45 ERROR(200) ZODB Couldn't load state for '\x00\x00\x00\x00\x00\x00\n\xa5' Traceback (innermost last): Module ZODB.Connection, line 533, in setstate TypeError: ('expected 1 arguments, got 0', <extension class AccessControl.cAccessControl.PermissionRole at 00E35258>, ()) I have spent quite some time debugging this last night. Line 533 in ZODB.Connection is state = unpickler.load() and the message means (it took some time to understand!) that unpickler.load() did not return anything. Also, once this happens, I find myself with two copies of my WebServiceProxy. From the extreme details of the cache: 1 '\x00\x00\x00\x00\x00\x00\n\xa5' 0 G Products.WebServiceProxy.WebServiceProxy.WebServiceProxy 3 '\x00\x00\x00\x00\x00\x00\n\xa5' 4 L Products.WebServiceProxy.WebServiceProxy.WebServiceProxy (google) which I guess is very bad from a consistency point-of-view. Well, I hope you're still reading me. Here are my questions: * Am doing something obviously wrong or stupid? * Is delattr(self, item.name+'__roles__') enough to remove all permission information about a method? The persistence error seems to be linked to a PermissionRole problem, which is usually what is contained in a m__roles__ field. * It is my understanding (after long analysis of the source) that what I'm doing with self._security is okay. I mean, it seems to me I should be able to redeclare my permission settings at run-time. Is this correct, or am I missing something? Acquisition is still rough to me. * Is it a good idea anyway to remove and add methods from a Product at run-time? (delattr(self, item.name), etc.)? I hope you will be able to give me some insights into this matter. I'd really like to be able to finish this Product somewhat, even as it is, it could have some value to the Zope community! :)

3 2