[Zope-dev] FTP interface being worked on?
John D. Heintz
jheintz@isogen.com
Mon, 19 Mar 2001 07:33:00 -0600
Chris McDonough wrote:
> Fred wrote:
>
>>> I guess I'm voting to rewrite this sentence:
>>>
>>> If this API is not implemented by the developer, the
>>
>> result is
>>
>>> a default serialized representation (perhaps XML
>>
>> pickle) on a
>>
>>> per-object basis
>>
>
> I think this makes sense.
>
>
>>> Maybe the issue is semantics. I think "potentially
>>
>> lossy" ==
>>
>>> "potentially leaky". Even a small leak would cause
>>
>> problems for us.
>>
>>> Maybe it wouldn't cause problems for others. But it
>>
>> sure seems like it
>>
>>> would be possible to create a solution that works for
>>
>> everyone. Namely,
>>
>>> a lossless representation that is easy to work with.
>>
>
> This is possible probably for many objects. DTML Methods,
> for instance, are basically just big bags of text with some
> security settings and other associated metadata. Recreating
> them losslessly from a filesystem representation is pretty
> easy. ZCatalogs, on the other hand, have lots of state,
> which is hard to adequately represent in anything but a
> morally binary representation. Maybe we won't even try to
> make it editable, and we'll choose to use XML for these.
>
> John wrote:
>
>> I'd first like to say that I applaud the goal stated in
>> the previous line!
>>
>> I think there are two key problems with achieving it.
>> 1) Because everyone writing extensions for Zope can
>> define their own
>> data structures it make it very difficult to store them
>> anywhere but an
>> object database. I think this problem has nearly the
>> same complexity as
>> figuring out the RDBMS table structures necessary for all
>> the Products
>> and builtin objects in Zope...
>
>
> Yes... luckily, because we have OO and polymorphism, we
> don't have to do this! ;-)
>
> I don't think it's reasonable or wise to impose any "master
> structure" for filesystem serialization of bodies of
> objects. Each instance (or perhaps each class) should
> define how best to serialize itself to disk.
> Representations between classes are likely to be radically
> different. A place for standardization is in the
> "properties" file(s) which accompany each object rep... this
> is likely to be XML or another structured variant.
My point was that serializing a Zope instance(class) in the general case
is about as hard as mapping to tables. As an example look at the
StructuredText format. This is a Zope class that has a very clear
serialized format, but a lot of work went into defining it and being
able to parse it. That amount of work may have to happen for each class
to be useful in serial format.
I think I might have missed some of your point though. If we
standardize "properties" to an XML file, then optionally dump other
files to expose specific aspects of an instance for serialized editing
it might not be as big a problem as I was thinking.
I guess I would suggest that the serialized form of a Zope instance by
default would be a single XML file, but that arbitrary sections of that
XML file could be custom dumped to separate serialized files with
similiar names. That way authors would have a pretty easy job of
overriding sections of the dump process to spit out one or more simple
files that have little parsing overhead.
>
>
>> 2) A lesser problem is when trying to edit the serialized
>> "files".
>> Because objects are methods and state how you modify an
>> object can be
>> guided if not controlled. When we have serialized the
>> objects in a Zope
>> system to files, we have exported only the state of the
>> objects in the
>> ZODB. We then have to live with the ability to foul up
>> invariant across
>> many objects by changing some data in the serialized
>> format. A good
>> example would be ZCatalogs. When some piece of data
>> changes the code
>> can automatically call reindex(), if I'm editing a file I
>> might not know
>> that I need to change other files due to runtime
>> dependencies.
>
>
> Yup... it's probably easiest to make ZCatalogs a black box.
Black box doesn't solve this problem, only the first one. Imagine that
I move a serialized version of a Zope object that is indexed by an
instance of ZCatalog (or many for that matter). When I move it the
ZCatalogs must be notified to handle the change, but only at import time
because ZCatalogs are serialized as binary for lots of good reasons.
Just listing that my example serialized file is used by some other
objects doesn't help because ZCatalog may not refer directly to the
object anyway.
The editing and import process must work together to track changed
files, moved files, and deleted files at a minimum.
This may not be good enough, because the code written into a Zope
Product may say that when property "x" is changed on these objects to
reindex the "foo" ZCatalog for that object. When I import the object
from the serialized format all I can know is that something changed, but
without expensive processing (XML diffing is hard in the general case,
we might be able to limit the structures to managable scope though) we
can't know that the "foo" ZCatalog should be updated instead of the
"bar" ZCatalog.
>
>
>> a) XML is structured enough that it can reliably hold the
>> data from the
>> ZODB. The current XML dump is not useful for this - it
>> would need to
>> create individual files and folders to represent
>> containment.
>
>
> This is pretty easy right now. Ten lines of recursive code
> can walk the whole tree if necessary and export only leaf
> objects.
>
>
>> b) A hybrid XML and custom dump solution. An Image for
>> example could
>> dump out as a binary image file with meta-data in a
>> similiarly name XML
>> file.
>
>
> Yes, each object should make its own policy regarding its
> body. Its metadata format should be standardized, however.
>
>
> _______________________________________________
> Zope-Dev maillist - Zope-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zope-dev
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope )
--
. . . . . . . . . . . . . . . . . . . . . . . .
John D. Heintz | Senior Engineer
1016 La Posada Dr. | Suite 240 | Austin TX 78752
T 512.633.1198 | jheintz@isogen.com
w w w . d a t a c h a n n e l . c o m