RE: Zope + Ape + Subversion (was: RE: [Zope-dev] Using a truely r evis ion based storage for Zope ?)
Shane, As Kapil already mentionned, svn's support for properties is quite flexible, so that doesn't worry me too much. I would imagine and administrator having the flexibility of choosing how objects get translateds to files might be handy. My initial, uneducated thoughts on the topic were simplistic, but then I'm a big K.I.S.S. fan: simply pickle the entire object back and forth as one entity. This means for each object, there is one file on the fs. The benefit is greater simplicity ... the downside is that you couldn't check those files out of subversion and interact with them. I also have to imagine the former is faster ? But if usnig the latter, then I'd think that in SVN there would be only 1 file, written by a specific mapper for a specific content type, say image. the data written to the file is such that if checked out, the file "works", so I can open it in photoshop or something. I however, would simply put everything else in properties, if that makes sense at all. Zope properties, security information, and so on. 1 porperty, 1 piece of data. No need for reading/writing multi-line text. The only thing I'm not so sure about is this "remainder" ... Though if it really is a base64 encoded string, there's no reason not to put that in a property as well. The benefits I see: - The filesystem reflects the object structure, no extra files lying around. - a more consistent way to setup mappers/gateways: there's "data", and there's everything else. Have a common/defined/smart way of handling "everything else". - with the use of the properties related svn funtions, we better leverage SVN's features. - possibly better performance/simpler code by not needing the whole multiple file management and read/write of text files. Heck the whole AnnotatedFileSystem, thing could be dropped/replaced with a PropertyFileSystem implementation of some kind maybe ?? What do you think ? J.F. -----Original Message----- From: Shane Hathaway [mailto:shane@zope.com] Sent: April 13, 2004 10:46 PM To: Jean-Francois.Doyon@CCRS.NRCan.gc.ca Cc: zope-dev@zope.org Subject: RE: Zope + Ape + Subversion (was: RE: [Zope-dev] Using a truely r evis ion based storage for Zope ?) On Tue, 13 Apr 2004 Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote:
Thanks for the extra tips, I'll check out those interfaces! I'm also getting up to speed on the whole mapper concept, where the work regarding properties handling seems to be ?
Ape supports both annotations and Zope properties. Annotations are blocks of multi-line text, while Zope properties are less constrained. On the filesystem, Ape stores many Zope properties together in a single annotation called "properties". Other annotations include object classification, the remainder pickle (encoded in base 64), and security information. What are the expectations and limits of Subversion properties? If they are blocks of multi-line text, you can store the annotations as Subversion properties. If not, we'll still need a .properties file. (Note that the name ".properties" is thus confusing. Oops.)
As for the seperation of code ... I now have a "subversion" directory as a sibbling to "fs" ... I had to edit a couple of files outside that directory, but still the seperation is nicer.
eventually I'd have to make it into a product, is that doable ?
Why not add your code to Ape? I think it would be easier for both of us to maintain that way. Shane
On Wed, 14 Apr 2004 Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote:
My initial, uneducated thoughts on the topic were simplistic, but then I'm a big K.I.S.S. fan: simply pickle the entire object back and forth as one entity. This means for each object, there is one file on the fs. The benefit is greater simplicity ... the downside is that you couldn't check those files out of subversion and interact with them. I also have to imagine the former is faster ?
If you store pickles, you can't merge, you can't store properties, in fact you can't even store folders as directories or keep track of references between objects. At that point, you'd do much better to just use FileStorage--KISS, after all. Ape's real utility is in breaking objects apart for storage. If you don't need that, you don't need Ape. But I like to think it makes the job of breaking objects apart for storage relatively simple.
But if usnig the latter, then I'd think that in SVN there would be only 1 file, written by a specific mapper for a specific content type, say image. the data written to the file is such that if checked out, the file "works", so I can open it in photoshop or something.
I however, would simply put everything else in properties, if that makes sense at all. Zope properties, security information, and so on. 1 porperty, 1 piece of data. No need for reading/writing multi-line text.
That's fine, although you'll need to distinguish Zope properties from other kinds of properties. For example, if some object has a class_name Zope property, you'd be in trouble if the system mixed that up with the class_name key of the object classification.
The only thing I'm not so sure about is this "remainder" ... Though if it really is a base64 encoded string, there's no reason not to put that in a property as well.
That's correct, it's always base64 encoded (with some comments.)
The benefits I see:
- The filesystem reflects the object structure, no extra files lying around.
Definitely.
- a more consistent way to setup mappers/gateways: there's "data", and there's everything else. Have a common/defined/smart way of handling "everything else".
I would recommend you defer this for a while. Lots of objects don't easily fit this model. For example, what is the "data" of a CMF tool? We can make plenty of progress on Subversion without thinking about changing the way mappers are defined.
- with the use of the properties related svn funtions, we better leverage SVN's features.
Yup.
- possibly better performance/simpler code by not needing the whole multiple file management and read/write of text files. Heck the whole AnnotatedFileSystem, thing could be dropped/replaced with a PropertyFileSystem implementation of some kind maybe ??
I'd guess there's no need for any such thing in Subversion. I think your code might eventually share a lot of code with the fs package, but if you'd like to start by making a clean break from the fs package, by all means go ahead. Shane
On Wed, 2004-04-14 at 19:14, Shane Hathaway wrote:
On Wed, 14 Apr 2004 Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote:
My initial, uneducated thoughts on the topic were simplistic, but then I'm a big K.I.S.S. fan: simply pickle the entire object back and forth as one entity. This means for each object, there is one file on the fs. The benefit is greater simplicity ... the downside is that you couldn't check those files out of subversion and interact with them. I also have to imagine the former is faster ?
If you store pickles, you can't merge, you can't store properties, in fact you can't even store folders as directories or keep track of references between objects. At that point, you'd do much better to just use FileStorage--KISS, after all. Ape's real utility is in breaking objects apart for storage. If you don't need that, you don't need Ape. But I like to think it makes the job of breaking objects apart for storage relatively simple.
But if usnig the latter, then I'd think that in SVN there would be only 1 file, written by a specific mapper for a specific content type, say image. the data written to the file is such that if checked out, the file "works", so I can open it in photoshop or something.
I however, would simply put everything else in properties, if that makes sense at all. Zope properties, security information, and so on. 1 porperty, 1 piece of data. No need for reading/writing multi-line text.
That's fine, although you'll need to distinguish Zope properties from other kinds of properties. For example, if some object has a class_name Zope property, you'd be in trouble if the system mixed that up with the class_name key of the object classification.
i don't think needs to much worrying about, just prefix zope properties with 'zope:' much the same way svn does with its properties 'svn:externals'
The only thing I'm not so sure about is this "remainder" ... Though if it really is a base64 encoded string, there's no reason not to put that in a property as well.
That's correct, it's always base64 encoded (with some comments.)
The benefits I see:
- The filesystem reflects the object structure, no extra files lying around.
Definitely.
- a more consistent way to setup mappers/gateways: there's "data", and there's everything else. Have a common/defined/smart way of handling "everything else".
I would recommend you defer this for a while. Lots objects don't easily fit this model. For example, what is the "data" of a CMF tool? We can make plenty of progress on Subversion without thinking about changing the way mappers are defined.
- with the use of the properties related svn funtions, we better leverage SVN's features.
Yup.
another thing to keep in mind here, svn is basically a versioned fs, and in addition to node/content history tracking and it has facilities/functions for apply deltas to content, and recieving delta diffs back etc, but these apply to node/content bodies and not to properties. ie. there are no facilties for diffing properties, and this would need to be implemented in python. -kapil
participants (3)
-
Jean-Francois.Doyon＠CCRS.NRCan.gc.ca -
Kapil Thangavelu -
Shane Hathaway