[Zope-dev] FTP interface being worked on?

[John D. Heintz]

Fred Wilson Horch wrote:

> Chris McDonough wrote:
> 
>> Fred Horch wrote:
>> 
>>> My major question is I don't understand the design
>>> decision to allow lossy representation. [...]
>>> 
>>> I think lossless serialization should be an explicit
>>> goal.  If a developer doesn't provide specific object serialization
>>> methods, then a default method (perhaps XML) should be invoked that is
>>> lossless even if hard to work with.
>> 
>> I think the proposal says something like this.
> 
> 
> The proposal states in part:
> 
>   If this API is not implemented by the developer, the result is
>   undefined (XML pickle representation if allowed by the object
>   on a per-object basis?).
> 
> I guess I'm voting to rewrite this sentence:
> 
>   If this API is not implemented by the developer, the result is
>   a default serialized representation (perhaps XML pickle) on a
>   per-object basis

> 
> 
>>> The whole point for us is to get full control of our
>>> objects through CVS.
>> 
>> That's one use, which is important to you.  Another is to
>> use Emacs or Dreamweaver on a representation of, for
>> example, DTML methods on a filesystem, which is important to
>> other folks.
>> 
>> The point of having potentially "lossy" representation of
>> objects is to make it easier to work with them in these
>> kinds of tools.  Nobody wants to edit XML, AFAICT.
> 
> 
> I see.  I agree with the goal to have a representation that is easy to
> work with in emacs.
> 
> Would it be possible to have a "lossless" representation that is also
> easy to work with?
> 
> The current XML export format is "lossless" but hard to work with.
> 
> 
>> "Potentially lossy" also doesn't mean "leaky".  It just
>> means that folks who expose their objects to this sort of
>> serialization can choose their own format, and if it
>> represents the object adequately for their own use in both
>> directions, it's good enough.
> 
> 
> Maybe the issue is semantics.  I think "potentially lossy" ==
> "potentially leaky".  Even a small leak would cause problems for us. 
> Maybe it wouldn't cause problems for others.  But it sure seems like it
> would be possible to create a solution that works for everyone.  Namely,
> a lossless representation that is easy to work with.

I'd first like to say that I applaud the goal stated in the previous line!

I think there are two key problems with achieving it.
1) Because everyone writing extensions for Zope can define their own 
data structures it make it very difficult to store them anywhere but an 
object database.  I think this problem has nearly the same complexity as 
figuring out the RDBMS table structures necessary for all the Products 
and builtin objects in Zope...

2) A lesser problem is when trying to edit the serialized "files". 
Because objects are methods and state how you modify an object can be 
guided if not controlled.  When we have serialized the objects in a Zope 
system to files, we have exported only the state of the objects in the 
ZODB.  We then have to live with the ability to foul up invariant across 
many objects by changing some data in the serialized format.  A good 
example would be ZCatalogs.  When some piece of data changes the code 
can automatically call reindex(), if I'm editing a file I might not know 
that I need to change other files due to runtime dependencies.
(I know that ZCatalog is a poor example because earlier in the thread 
cataloging was discussed wrt lossy/lossless behavior, but it was the 
easiest for me to make my point with.)

Having said that, I suspect that a few systems could solve the first 
problem.  (I don't know how to solve the second one with serialized data...)

a) XML is structured enough that it can reliably hold the data from the 
ZODB.  The current XML dump is not useful for this - it would need to 
create individual files and folders to represent containment.

b) A hybrid XML and custom dump solution.  An Image for example could 
dump out as a binary image file with meta-data in a similiarly name XML 
file.

c) A special file system - like ReisferFS might be that system.  To my 
knowledge ReisferFS is eventually intended to be a melding of file 
system, relation db, and object db.  This is obviously serious 
paraphrasing, but I think it may have enough descriptive power to 
replace XML.  I know know how this would interact with CVS, or be 
edited, but I thought I'd mention it.

None of these solve problem 2), but then again I don't think anything does..

Thanks for listening,
John

> 
> I completely agree with the point that we want to be able to work with
> representations of objects using tools like emacs and dreamweaver.  In
> fact, we'd like to use emacs as our front end to CVS.  The ideal
> situation would be to edit Zope objects in emacs, publish them to a Zope
> object database, test them (perhaps using a separate web browser like
> Netscape or Internet Explorer), and then once everything is working,
> commit the objects to a CVS repository (using emacs or from the command
> line).
> 
> 
>> If you want a lossless "morally binary" representation, it's
>> probably best to use XML export, which is great for your
>> purposes, because it already exists!  ;-)
> 
> 
> I've heard it said that all progress is due to the unreasonable man.  So
> to do my part for progress, what I want is a lossless "morally plain
> text" representation. ;-)
> 
> If the existing XML export really was great for our purposes, I'd be
> done!  The problem is that everything comes out in one big monolithic
> file.  That's not good for project management using CVS.  (At least, as
> far as I can tell.)
> 
> I think there is the potential for a really good solution that solves
> our need to manage our project via CVS, and to solve the greater need to
> enhance the Zope management interface to support tools that work with
> filesystem objects.
> 
> I am about to jump into this project next week.  I do want to stay in
> touch with anyone who is working on similar projects, so please keep in
> touch!  I will post reports as I make progress in case anyone is
> interested.
> 
> Thanks,
> Fred



-- 
. . . . . . . . . . . . . . . . . . . . . . . .

John D. Heintz | Senior Engineer

1016 La Posada Dr. | Suite 240 | Austin TX 78752
T 512.633.1198 | jheintz@isogen.com

w w w . d a t a c h a n n e l . c o m