June 2000 - Zope-Dev

BerkeleyStorage with ZODB problem
by nw_moriarty 01 Jun '00

01 Jun '00

I'm developing a large program and think that ZODB with BerkeleyStorage is the best option. I had a python test program working using FileStorage that worked. I substituted BerkeleyStorage and it fails thus: Traceback (innermost last): File "btest.py", line 118, in ? mydb = Database() File "btest.py", line 22, in __init__ self.db = DB(SleepyStorage(file)) File "/usr/local/dc/zope/lib/python/ZODB/BerkeleyStorage.py", line 107, in __init__ self._index.open(name, db.DB_BTREE, db.DB_CREATE | dothread) File "/usr/lib/python3rd/TCS/bsddb/db.py", line 639, in open val = apply(dbc.DB_open,(self.this,arg0,)+args) db.error: (22, 'Invalid argument') Has any got a working example of BerkeleyStorage that works! Nigel --== Sent via Deja.com http://www.deja.com/ ==-- Before you buy.

1 0

Zope 2.1.6: Test.py problem with patch
by Dieter Maurer 01 Jun '00

01 Jun '00

Problem: Description: If a DTML method is accessed with "ZPublisher/Test.py", an "Unauthorized" exception is raised in "verify_watermark". Environment: Zope 2.1.6, Linux 2, Intel Analysis: The module "BaseRequest" is imported twice: once as "BaseRequest" and once as "ZPublisher.BaseRequest". The AUTHENTICATED_USER (and its watermark) is constructed by the first module instance; the check is made against the watermark of the second module instance and therefore fails. Patch: URL:http://www.handshake.de/~dieter/pyprojects/zope/test.pat Dieter

1 0

anyone have a ZOracleDA Setup-8.1.6-linux-intel
by Shawn Murphy 01 Jun '00

01 Jun '00

In other words...I am trying to get the ZOracleDA compiled for RedHat Linux for Zope 2.1.6, Oracle 8.1.6 and ZOracleDA-2.2.0b1, but am having problems creating a Setup file which does the job. It seems like others have had some success in this area. Anybody able to share some tips or better yet, the 'Setup-8.1.6-linux-intel' that they created. On a slightly different note, is the ZOracleDA source in a public CVS somewhere? Maybe the solution can be found there or I would be happy to place it there once developed. I can't find the ZOracleDA in http://classic.zope.org/cgi-bin/cvsweb.cgi/Zope2/ Thanks All! ==================================================================== Shawn Murphy Research and Development mailto://smurp@emergence.com Emergence by Design work://1.780.413.6397 http://www.emergence.com

2 1

Ownership changes for 2.2 beta...
by Brian Lloyd 01 Jun '00

01 Jun '00

Hi all - There was a recent discussion regarding the behavior of ownership, particularly regarding the setting of ownership for move, rename and import operations. I've just finished checking in some changes to the model in alpha 1 that will be in the upcoming beta, along with the updated documentation (help system). The changes basically are: o Ownership information is not changed for move or rename operations. o Import now has a radio button so that you can elect to import objects with their existing ownership info intact. This is in CVS now for those who are interested - I've also included the updated help text with this email for review. updated help for ownership: Ownership Overview This view allows you to manage the ownership of a Zope object. All Zope objects except objects within the Control Panel support ownership. When an object is created, copied or imported in Zope, the logged-in user performing the operation becomes the owner of the resulting object(s). Note that simply moving or renaming an object does not change its ownership. Ownership is most important for Zope objects that "execute" content supplied through the web, such as DTML Methods, DTML Documents and SQL Methods. The abilities of these executable objects are constrained by the abilities of the object's owner as well as the user causing the object to execute from a Zope security point of view. In other words, an executable cannot perform operations that its owner could not perform directly. While most Zope objects support ownership, objects may not always have an owner assigned. Objects that do not have an assigned owner are unowned. An unowned executable object executes with the permissions of the user running the object. There are three possible states of ownership that a Zope object may be in. The object may be unowned, it may be owned directly or it may be owned implicitly. If an object is owned directly, the ownership information is stored in the object itself. If an object is owned implicitly, it acquires its ownership information from its container (which may in turn acquire its ownership information from its container, and so on). When Zope automatically assigns ownership to newly created or copied objects, it tries to use implict ownership if possible. This makes it easier to change the ownership of many objects at one time without having to visit each object directly. For example, if everything in the folder research is owned by Fred and you want to change the ownership of everything in the folder to Jim, you only need to visit the research folder and change the ownership there if all of the objects under research are owned implicitly. Managing ownership Clicking on the Ownership tab of a Zope object will display the id and title of the object along with the current ownership status of the object. If the object is not owned, you will see a message like: Zope QuickStart Outline (QuickStart) is unowned. If you have the Take ownership permission for the object, you will also see a button labeled Take ownership. Click the button to take ownership of the object. If the object is owned directly, you will see a message like: My Folder (my_folder) is owned directly by fred (acl_users). You will also see a button labeled Make ownership implicit (acquired). Click this button to remove the direct ownership information in the object and have the object acquire its ownership information from its container. If you have the Take ownership permission for the object, you will also see a button labeled Take ownership. Click the button to take ownership of the object. If the object is owned implicitly, you will see a message like: My Folder (my_folder) is owned indirectly (through acquisition) by fred (acl_users). You will also see a button labeled Make ownership explicit . Click this button to make the object store the ownership information that it is currently acquiring in itself directly. After making ownership explicit, changes to the ownership information in the current object's containers will no longer affect the ownership information of the current object. If you have the Take ownership permission for the object, you will also see a button labeled Take ownership. Click the button to take ownership of the object. Notes on Ownership It is possible to remove a user in your Zope installation while objects that were owned by that user remain (for instance, someone in your organization moves on, so you remove his account on your Zope server). When you remove a Zope user, objects that were owned by that user still have ownership information that refers to that user. This will not cause an error, but it is important to understand what happens to Zope objects whose owners no longer exist. This is most important for "executable" objects. Usually, the abilities of executable objects are constrained by the abilities of the object's owner as well as the user causing the object to execute. If Zope cannot find the user that the executable's ownership information refers to when it tries to execute the object, it will use the special nobody or anonymous user instead. The nobody user has very minimal priveleges, so you should take this into account when deciding what to do about a user's owned objects when you delete the user. If the user had DTML documents or other executable objects that depended upon their owner to have a higher level of privilege than the nobody user, they will not work correctly until their ownership has been reassigned to another user with sufficient privileges. updated help for import/export: Folder>Import/Export This view allows you to import and export objects from the Zope database. To export an object specify its id in the Export object id field and click the Export button. If you choose Download to local machine, the export file will be downloaded by your browser, if you choose Save to file on server, the export file will be saved in the Zope var directory of the server on which Zope is running. If you choose XML format, the export file will be encoding as XML, otherwise it will be encoded as a binary file. To import an object into to Zope, specify the name of the import file in the Import file name field. The import file should be located in the Zope import directory of the Zope installation. Click the Import button to import the objects in the import file. By default, you will become the owner of the objects you import. In some cases you may want to preserve the ownership information in the imported objects. To preserve the existing ownership information in the imported objects, select the retain existing ownership information option before clicking the Import button. Note that when you elect to retain existing ownership information, the ownership information will be used exactly as it appears in the imported objects. For instance, if the object you are importing was acquiring its ownership information in the place where you exported it, it will continue to acquire its ownership information when you import it (though the actual ownership information may now be different in the context into which it is imported). Likewise, if the ownership of an object was explicit (not acquired) when it was exported, then it will be explicit after it is imported. The important thing to note in this case is that if you export an object from another Zope site, the user who owned that object in the other site may not exist in the site you are importing the object into. Zope does not check to make sure the owner exists if you import an object that was explicitly owned. If the explicitly named owner does not exist in the site that you are importing into, imported objects that depended on their owner having a higher level of privilege than the special nobody user may not work correctly until their ownership has been reassigned to another user with sufficient privileges. See the online help for ownership for more details. Brian Lloyd brian(a)digicool.com Software Engineer 540.371.6909 Digital Creations http://www.digicool.com

1 0

How Medusa handles file extensions
by Chimezie Thomas-Ogbuji 01 Jun '00

01 Jun '00

Does any one know how the Medusa FTPServer determines which Zope Object/Product to instanciate based on the file extension of a newly created file? For instance if I create a .dtml file, a DTMLMethod is created, if I create a .gif file a Image is created, and any unrecognized extension is instanciated as a File object. I'm trying to capture files with .xml extensions so they are instanciated as XMLDocuments -- Chimezie Thomas-Ogbuji Consultant Fourthought Inc. (303) 583 9900 ext 104 cogbuji(a)fourthought.com

1 0

Re: [Zope-dev] [announce] for the CVS junkies: multiple ZODB mounting product.
by Anthony Baxter 01 Jun '00

01 Jun '00

>>> Shane Hathaway wrote > My feeling is that _no_one_ should ever have the Manager role in a user > folder besides the root user folder. The problem with that, of course, > is that creating managers is just easier. ;-) *nod* I've been, ahem, a little guilty of that. At least 2.2 will stop me just logging in as superuser "just because it's easier". > > I think I'm going to need to revisit the code tomorrow morning and write > > the little 'ZODB maker'. > > I'll leave the decisions to your capable hands now. I'd bet many > Zopistas are grateful already! Hey! How'd I end up holding this small baby? Dammit, I think it just made a mess. Oh well, teach me to fiddle with stuff. :) New version of the MountedFileStorage product is up, and cataloged. See http://www.zope.org/Members/anthony/software/MountedFileStorage I'll put a note about it on zope(a)zope.org. Anthony -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

1 0

Re: [Zope-dev] [announce] for the CVS junkies: multiple ZODB mounting product.
by Anthony Baxter 01 Jun '00

01 Jun '00

>>> Shane Hathaway wrote > Okay, I looked a little more and I see you're not allowing slashes in > the filename. That's a decent solution. Don't forget backslashes (for > MS Windows). Bah. Silly backwards operating system. Ok dokey. > You may want to create mounted databases in a directory > other than /var, since there may be other sensitive files there. I thought about that, then thought 'that can wait for another day'. This is muchmuch simpler to implement. > If you're using more than one acl_users folder, remember that the > Manager role, regardless of user context, will give users the right to > use your product in their own directory. Hm - it's not possible for a Manager in a higher level to remove privileges from a manager defined in a subfolder, I think. A pity. Will document it. > One small change you should make: call the MountPoint._test() method > right after creating the new mount point. It will ensure any mount > exceptions are propagated up correctly. Ok dokey. I've just made a change to my code for both of these, > > with each partner object a different ZODB. Unfortunately, the current > > implementation uses ZClasses in each one. So if I upgrade any one, I > > have to upgrade all of them. (and then we're back to the old 'how do > > you upgrade ZClasses' issue, which, unless I'm mistaken, has never > > really been resolved). > > AFAIK you can just delete or rename a product and install the new > version of a set of ZClasses. So long as the classid stays the same. There's also a scaling issue - if I have dozens of objects, this will be bad. I've got a solution for this case - the stuff that goes into the sub-ZODB will be redone to only use python-created objects. > ZODB/Mount.py already provides such a registry, so you may want to use > that, although it applies to *all* mount points rather than some kind > of subset. Yeah, I was going to try and hook a mounted databases tab into the Control Panel. Zope gets an /etc/mtab :) > Actually FileStorage provides a nifty trick: you can just name a > database file and if it doesn't exist it will be created. That could > be a security problem. Or it might just be a good feature. It will be > created with the "true" bare minimum, which doesn't even have an > Application object Yep, I discovered that. Since it breaks without an Application, and it might be a security problem, I just disabled it - the file must exist first. > The id of the MountPoint and the last item of the mounted path must > match for a subtle reason. When ObjectManager looks up the object, it > uses the ID of the mount point. The object it finds is assumed to have > the correct ID. If there is no assurance of that, mounted databases > could cause some confusion for OFS. I also hit this. Much confusion on the part of poor little Zope. The code now will only let you specify an 'id' if you're mounting the root of a ZODB (since the default ZODB doesn't have an id on it's root object.) This may not be the right thing to do, but I couldn't figure out an appropriate way to do it otherwise. Hm, maybe a small script in utilities that will create a real bare minimum ZODB with an Application and a Control_Panel &c. might be the go here. > One possibility you may consider is allowing the user to create a > FileStorage on the fly. You can do one of two things: override > _getMountRoot(), which is a hook I provided for the very purpose of > accessing databases that have no Application object, or manually create > an Application object in the new database. Either way, you must ensure > that the ID of the mounted object is the same as the ID of the mount > point. I think I'm going to need to revisit the code tomorrow morning and write the little 'ZODB maker'. > Hmm... I don't see why you would want to mount the Control_Panel, > except to fiddle with ZClasses. I would suggest finding a way to get > ZClasses to operate more naturally might be a better approach. Yep, you figured out what I was doing - it was more a matter of 'should it work this way?' -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

2 1

Re: [Zope-dev] [announce] for the CVS junkies: multiple ZODB mounting product.
by Anthony Baxter 01 Jun '00

01 Jun '00

>>> Shane Hathaway wrote > You did better than you think, Anthony. I was going to release the > product I've been putting together, but your little product does > virtually the same thing. Good work! Lucky guessing, and trawling CVS. :) > There is, however, a major security problem with making it so easy to > mount a FileStorage. In my product, per Jim's suggestion I set it up > so that the user is required to create an External Method in order to > mount a database. That actually makes things a little more flexible in > that it provides you the ability to mount things other than > FileStorages. I'm curious - could you elaborate on the security issues of this? The convenience of not having to write an External Method each time you want to mount or unmount a ZODB is high. :) > There are still some issues with mounted databases. The primary issue > is that they interact strangely with ZClasses. However, if you have > the same ZClass defined in both the primary database and the mounted > database, ZClasses seem to work fine. (I wouldn't bet my company on > it, though.) Yah, I've been thinking a bit about that, and it's not at all obvious how it should work. In my case, I want to be able to mount something like /partners/partnerA partnerB partnerC with each partner object a different ZODB. Unfortunately, the current implementation uses ZClasses in each one. So if I upgrade any one, I have to upgrade all of them. (and then we're back to the old 'how do you upgrade ZClasses' issue, which, unless I'm mistaken, has never really been resolved). > The other issues are that you can't easily undo nor manage mounted > databases. That is more of a UI problem which could be solved by > product authors, however. I envision a product that would create a > "mounted database manager" object. Inside the database manager you > could add as many mount points as you want. The database manager > object is where you would go to pack any of the mounted databases. Yeah - the mount points are entirely invisible. One thing I'd considered doing was to make the Product provide a registry of mount points and their underlying FileStorages. > ... And please report any problems which you may think are being caused > by the mount. http://www.zope.org/Wikis is a mounted database and it's > running fine, but YMMV. It is? Okay, that makes me feel better. A couple of issues I'd like clarification on: Any mounted filestorage is searched for an object called 'Application'. I'm assuming it will also want things like Control_Panel &c as well. What's the bare minimum a new mounted FS needs? Right now, I'm copying Data.fs.in and blowing away the stuff in it, but nuking it's standard_html_blah is a bit more problematic. Would it be worth making a 'create new empty ZODB FS' function? If you try to mount /Control_Panel/Products from a mounted ZODB, you get an 'Attribute Error: _owner'. I'm assuming the Control Panel is special cased by the ownership code when it's the main ZODB. How does this work with the Owner mods in recent CVS? If I remember correctly, objects keep a reference to their Owner - won't this break across ZODBs? thanks, Anthony

2 1

[announce] for the CVS junkies: multiple ZODB mounting product.
by Anthony Baxter 01 Jun '00

01 Jun '00

I just bolted together from bits I scanned from CVS a 'MountedFileStorage' product. This allows you to use multiple ZODBs in a single Zope. It's something I've done "just for the hell of it" but thought that there might be other people out there who want to play with this stuff. http://www.zope.org/Members/anthony/software/MountedFileStorage_hack/ It needs an up-to-date CVS. As for why I've done this - I _really_ _really_ want multiple ZODB support - we have a couple of requirements for it. This was an excuse to fiddle and figure out how it works. Please don't run this on a Zope which hasn't got a backed-up copy of the Data.fs. Anthony

2 1

Re: [Zope-dev] Memory 2.1.4-2.1.6
by Eric Sattler 01 Jun '00

01 Jun '00

Bill Anderson wrote: > > Eric Sattler wrote: > > > > Bill, > > > > Jason and I seem to have gotten mixed up here. > > Yeah, mix-ups tend to happen on lists ;-) > > > Here is my > > configuration and situation, the 2.1.4-2.1.6 means that it occurs > > under both versions of Zope. It also occurs with the Neil Schemenauer > > gc python patches. Here is my original posting: > > > > I also am having severe problems with memory creep. Our problem > > exhibits itself by quickly using gobs of memory and requiring a > > zope restart after about a day and a half at about 100M resident. > > > > First our configuration: > > > > Core Components: > > > > FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course > > Postgres 6.5.3/Postgres 7.0(fixes some memory leaks) > > Python 1.5.2 > > Zope 2.1.4/2.1.6 have tried both > > pcgi with ZServer > > Apache w/mod_ssl 1.3.9 and 1.3.12 > > > > Products: > > > > ZPySQLDA > > SQLSession > > GenericUserFolder > > I don't know if you saw it or not, but so far GUF has been determined to > be a memory leak, and further was dermined to be causing further leakage > in SQLSession (as well as on or two other products, IIRC). A patch/new > version was posted, and may be worth a trial upgrade to see if that > fixes it. IIRC, it was related to things not being removed from cache, > imagine that! :) > Did see that, sounds like some promising progress has been made. Is the latest version of SQLSession up on the Zope site? I think Anthony Baxter referred to releasing a 0-2-9 version? Can't seem to find anything greater than 0-2-3, or else there is a patch directory I am missing. > To do this without compromising your production site, you can basically > copy your setup to the other machine (remember that the paths should be > kept the same if possible), carry out the experiments, and note the > results. This is how I test many things. Got several machines with 3 different versions of zope on 'em ready once I install the patches for GUF and SQLSession.

1 0