Fred Wilson Horch wrote:
Chris McDonough wrote:
Fred Horch wrote:
My major question is I don't understand the design decision to allow lossy representation. [...]
I think lossless serialization should be an explicit goal. If a developer doesn't provide specific object serialization methods, then a default method (perhaps XML) should be invoked that is lossless even if hard to work with.
I think the proposal says something like this.
The proposal states in part:
If this API is not implemented by the developer, the result is undefined (XML pickle representation if allowed by the object on a per-object basis?).
I guess I'm voting to rewrite this sentence:
If this API is not implemented by the developer, the result is a default serialized representation (perhaps XML pickle) on a per-object basis
The whole point for us is to get full control of our objects through CVS.
That's one use, which is important to you. Another is to use Emacs or Dreamweaver on a representation of, for example, DTML methods on a filesystem, which is important to other folks.
The point of having potentially "lossy" representation of objects is to make it easier to work with them in these kinds of tools. Nobody wants to edit XML, AFAICT.
I see. I agree with the goal to have a representation that is easy to work with in emacs.
Would it be possible to have a "lossless" representation that is also easy to work with?
The current XML export format is "lossless" but hard to work with.
"Potentially lossy" also doesn't mean "leaky". It just means that folks who expose their objects to this sort of serialization can choose their own format, and if it represents the object adequately for their own use in both directions, it's good enough.
Maybe the issue is semantics. I think "potentially lossy" == "potentially leaky". Even a small leak would cause problems for us. Maybe it wouldn't cause problems for others. But it sure seems like it would be possible to create a solution that works for everyone. Namely, a lossless representation that is easy to work with.
I'd first like to say that I applaud the goal stated in the previous line! I think there are two key problems with achieving it. 1) Because everyone writing extensions for Zope can define their own data structures it make it very difficult to store them anywhere but an object database. I think this problem has nearly the same complexity as figuring out the RDBMS table structures necessary for all the Products and builtin objects in Zope... 2) A lesser problem is when trying to edit the serialized "files". Because objects are methods and state how you modify an object can be guided if not controlled. When we have serialized the objects in a Zope system to files, we have exported only the state of the objects in the ZODB. We then have to live with the ability to foul up invariant across many objects by changing some data in the serialized format. A good example would be ZCatalogs. When some piece of data changes the code can automatically call reindex(), if I'm editing a file I might not know that I need to change other files due to runtime dependencies. (I know that ZCatalog is a poor example because earlier in the thread cataloging was discussed wrt lossy/lossless behavior, but it was the easiest for me to make my point with.) Having said that, I suspect that a few systems could solve the first problem. (I don't know how to solve the second one with serialized data...) a) XML is structured enough that it can reliably hold the data from the ZODB. The current XML dump is not useful for this - it would need to create individual files and folders to represent containment. b) A hybrid XML and custom dump solution. An Image for example could dump out as a binary image file with meta-data in a similiarly name XML file. c) A special file system - like ReisferFS might be that system. To my knowledge ReisferFS is eventually intended to be a melding of file system, relation db, and object db. This is obviously serious paraphrasing, but I think it may have enough descriptive power to replace XML. I know know how this would interact with CVS, or be edited, but I thought I'd mention it. None of these solve problem 2), but then again I don't think anything does.. Thanks for listening, John
I completely agree with the point that we want to be able to work with representations of objects using tools like emacs and dreamweaver. In fact, we'd like to use emacs as our front end to CVS. The ideal situation would be to edit Zope objects in emacs, publish them to a Zope object database, test them (perhaps using a separate web browser like Netscape or Internet Explorer), and then once everything is working, commit the objects to a CVS repository (using emacs or from the command line).
If you want a lossless "morally binary" representation, it's probably best to use XML export, which is great for your purposes, because it already exists! ;-)
I've heard it said that all progress is due to the unreasonable man. So to do my part for progress, what I want is a lossless "morally plain text" representation. ;-)
If the existing XML export really was great for our purposes, I'd be done! The problem is that everything comes out in one big monolithic file. That's not good for project management using CVS. (At least, as far as I can tell.)
I think there is the potential for a really good solution that solves our need to manage our project via CVS, and to solve the greater need to enhance the Zope management interface to support tools that work with filesystem objects.
I am about to jump into this project next week. I do want to stay in touch with anyone who is working on similar projects, so please keep in touch! I will post reports as I make progress in case anyone is interested.
Thanks, Fred
-- . . . . . . . . . . . . . . . . . . . . . . . . John D. Heintz | Senior Engineer 1016 La Posada Dr. | Suite 240 | Austin TX 78752 T 512.633.1198 | jheintz@isogen.com w w w . d a t a c h a n n e l . c o m