Chris McDonough wrote:
Fred wrote:
I guess I'm voting to rewrite this sentence:
If this API is not implemented by the developer, the
result is
a default serialized representation (perhaps XML
pickle) on a
per-object basis
I think this makes sense.
Maybe the issue is semantics. I think "potentially
lossy" ==
"potentially leaky". Even a small leak would cause
problems for us.
Maybe it wouldn't cause problems for others. But it
sure seems like it
would be possible to create a solution that works for
everyone. Namely,
a lossless representation that is easy to work with.
This is possible probably for many objects. DTML Methods, for instance, are basically just big bags of text with some security settings and other associated metadata. Recreating them losslessly from a filesystem representation is pretty easy. ZCatalogs, on the other hand, have lots of state, which is hard to adequately represent in anything but a morally binary representation. Maybe we won't even try to make it editable, and we'll choose to use XML for these.
John wrote:
I'd first like to say that I applaud the goal stated in the previous line!
I think there are two key problems with achieving it. 1) Because everyone writing extensions for Zope can define their own data structures it make it very difficult to store them anywhere but an object database. I think this problem has nearly the same complexity as figuring out the RDBMS table structures necessary for all the Products and builtin objects in Zope...
Yes... luckily, because we have OO and polymorphism, we don't have to do this! ;-)
I don't think it's reasonable or wise to impose any "master structure" for filesystem serialization of bodies of objects. Each instance (or perhaps each class) should define how best to serialize itself to disk. Representations between classes are likely to be radically different. A place for standardization is in the "properties" file(s) which accompany each object rep... this is likely to be XML or another structured variant.
My point was that serializing a Zope instance(class) in the general case is about as hard as mapping to tables. As an example look at the StructuredText format. This is a Zope class that has a very clear serialized format, but a lot of work went into defining it and being able to parse it. That amount of work may have to happen for each class to be useful in serial format. I think I might have missed some of your point though. If we standardize "properties" to an XML file, then optionally dump other files to expose specific aspects of an instance for serialized editing it might not be as big a problem as I was thinking. I guess I would suggest that the serialized form of a Zope instance by default would be a single XML file, but that arbitrary sections of that XML file could be custom dumped to separate serialized files with similiar names. That way authors would have a pretty easy job of overriding sections of the dump process to spit out one or more simple files that have little parsing overhead.
2) A lesser problem is when trying to edit the serialized "files". Because objects are methods and state how you modify an object can be guided if not controlled. When we have serialized the objects in a Zope system to files, we have exported only the state of the objects in the ZODB. We then have to live with the ability to foul up invariant across many objects by changing some data in the serialized format. A good example would be ZCatalogs. When some piece of data changes the code can automatically call reindex(), if I'm editing a file I might not know that I need to change other files due to runtime dependencies.
Yup... it's probably easiest to make ZCatalogs a black box.
Black box doesn't solve this problem, only the first one. Imagine that I move a serialized version of a Zope object that is indexed by an instance of ZCatalog (or many for that matter). When I move it the ZCatalogs must be notified to handle the change, but only at import time because ZCatalogs are serialized as binary for lots of good reasons. Just listing that my example serialized file is used by some other objects doesn't help because ZCatalog may not refer directly to the object anyway. The editing and import process must work together to track changed files, moved files, and deleted files at a minimum. This may not be good enough, because the code written into a Zope Product may say that when property "x" is changed on these objects to reindex the "foo" ZCatalog for that object. When I import the object from the serialized format all I can know is that something changed, but without expensive processing (XML diffing is hard in the general case, we might be able to limit the structures to managable scope though) we can't know that the "foo" ZCatalog should be updated instead of the "bar" ZCatalog.
a) XML is structured enough that it can reliably hold the data from the ZODB. The current XML dump is not useful for this - it would need to create individual files and folders to represent containment.
This is pretty easy right now. Ten lines of recursive code can walk the whole tree if necessary and export only leaf objects.
b) A hybrid XML and custom dump solution. An Image for example could dump out as a binary image file with meta-data in a similiarly name XML file.
Yes, each object should make its own policy regarding its body. Its metadata format should be standardized, however.
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
-- . . . . . . . . . . . . . . . . . . . . . . . . John D. Heintz | Senior Engineer 1016 La Posada Dr. | Suite 240 | Austin TX 78752 T 512.633.1198 | jheintz@isogen.com w w w . d a t a c h a n n e l . c o m