Re: [Zope] [FEATURE REQUEST] comment at the top of DTML Methods and Documents
On Thu, May 24, 2001 at 04:02:49PM -0500, Steve Drees wrote:
Perhaps you could modify your dump command to get the meta type Write it out in a <dtml-comment></dtml-comment> And then dump the results of document_src() This wouldn't require a change to Zope. Just to ZShell
Of course I could do that in approximately 5 minutes. However the dump or even ZShell is not the problem, this was just a bad example. Say I edit some DTML Method from scratch with emacs (or any other tool which don't know anything about Zope), now I want to put it into the ZODB, but not using the zope's management interface, for example using ZShell's wget command or another yet to be released tool (or maybe FTP I don't know) Then during the upload, Zope can't decide automatically what the object meta type will be, and will make a guess which is bad in 50% of the cases: either it's a DTML Method, or it's a DTML Document, and both will be detected with a text/html mime type. talking about information redundancy, please explain why a Python Script contains it's meta type in a comment at its top ! It has no meaning to Zope, because you can't upload and use other types of Python source inside Zope (you don't upload products or external methods), and it has no meaning to Python or any text editor out there. That's why I think my suggestion is just as bad as this Python Script comment: Why don't you do it in DTML objects too ? (to push it further: uploading a structured text document, the mime type will be recognized as text/plain anyway, and the meta type will be File, I mean when not using the ZMI)
Then you could modify wget to make use of this comment when importing.
Again, it's a five minutes job to modify ZShell to do that, but I wanted a more general solution in order for everybody to be able to use it with non Zope-aware software. Hoping this clarifies what i want. bye, Jerome Alet
Jerome Please correct me if I misunderstand your idea and its scope.. 1. You want to embed meta_data header comments inside Zope files just as Python encourages doc strings 2. This is allows much more flexible Zope workflow because a wider arrange of tools and paradigms can be used: ZODB/Zope ones as well as external flat-file style apps [Editors, Dreamweaver, wxyzIDE, ZShell, Smart-Clients, Flash front ends, XML-RPC tools, etc..] 3. These embedded meda_type data comments wodl faciliate meta_methods = a fancy name for methods aimed at auto-processing and handling of filse which inlcude this information. ...It sounds wonderful to me. Over time this could help to make Zope much more self-administering. Especially as it extends to include more media filetypes, special documents, XML formats etc.. Zope's basis as a publishing framework really begs to go beyond its own current meta_types. But the process still seems way too adhoc and need too much manual attention. Thre are some contexts where one might want ot store media fiesl in the ZODB up to cettain size or wher they are known to be static, while at a certain point they need to migrate to LocalFS, ExtFile/Image etc. Documents are the same. One might want to initialize a site with dummy 'stub' files and document types, then use 'meta_methods' to load and manage the current data/media, based on higher level custom programs and conditions. An interesting example of a Product which would help and be helped is DocumentLibrary: http://www.zope.org/Members/Kaivo/DocumentLibrary ./Jason
From: "Jason Cunliffe" <jasonic@nomadicsltd.com>
Thre are some contexts where one might want ot store media fiesl in the ZODB up to cettain size or wher they are known to be static, while at a certain point they need to migrate to LocalFS, ExtFile/Image etc. Documents are the same.
I think this is heading in the wrong direction.... ZODB isn't magic, and neither is Persistence, they're both controllable. There was a discussion a while back about 'fine grained pack' of the ZODB -- that would really be a great feature, and it seems there's some work going on in that area. Perhaps each object could have a pack(date) method. But aside from that, the only clear need I've seen for external storage is external (outside zope) access, which is a requirement independent of size or number of files...
Thre are some contexts where one might want ot store media fiesl in the ZODB up to cettain size or wher they are known to be static, while at a certain point they need to migrate to LocalFS, ExtFile/Image etc. Documents are the same.
I think this is heading in the wrong direction.... ZODB isn't magic, and neither is Persistence, they're both controllable. There was a discussion a while back about 'fine grained pack' of the ZODB -- that would really be a great feature, and it seems there's some work going on in that area. Perhaps each object could have a pack(date) method. But aside from that, the only clear need I've seen for external storage is external (outside zope) access, which is a requirement independent of size or number of files...
Please clarify for me then.. ZODB is best when: - There are more READs than WRITEs. Because of 'Undo', WRITE intensive designs will swell ZODB too quickly External Storage is best - There are frequent WRITEs. - Very Large Files, or files which start off small and become very large later, [Digital Video for example]. These couild more quickly combine to tip ZODB past 2Gb limit [not a problem everywhere] - Sharing data with others applications, sharing with Apache, whatever - Need to leverage directly the filesystem's permissions [Linux & family] What do you think? ./Jason
From: "Jason Cunliffe" <jasonic@nomadicsltd.com>
Please clarify for me then..
Just my $0.002...
ZODB is best when: - There are more READs than WRITEs.
True of not only of any data base, but most file systems (the only notable exception I can think of is TAPE).
Because of 'Undo', WRITE intensive designs will swell ZODB too quickly
Not necessarily. You can make 'volatile' storage and control how it's put into the ZODB -- take a look at the source for the ThreadSafeCounter product for an example.
External Storage is best - There are frequent WRITEs.
See above... keep in mind, data.fs is stored in the same file system, so it has the same performance characteristics as any other file (and some opportunities for improvements as well).
- Very Large Files, or files which start off small and become very large later, [Digital Video for example]. These couild more quickly combine to tip ZODB past 2Gb limit [not a problem everywhere]
...should never be a problem -- see http://www.zope.org/Members/hathawsh/PartitionedFileStorage
- Sharing data with others applications, sharing with Apache, whatever
This is the real issue. By apache, I suppose you mean passing absolute URLs to apache to serve directly out of the file system. I suspect just using Zserver with Squid would equal the performance, but haven't tested it. But, let's say you have external data sources and sinks for these files, like legacy data bases, or some such. Some of those could probably be linked via some database adapters, python code, etc., but for some situations, it would make more sense to use external storage.
- Need to leverage directly the filesystem's permissions [Linux & family]
I'm not sure what you mean... there is some stuff around for LDAP and some others, so you could create a 'parallel universe' of permissions inside zope... right? So then you could go either way with the file storage.
What do you think?
Personally... you can tell I'm keen on ZODB, and think it's worth working with rather than around...
On Thu, May 24, 2001 at 10:52:22PM -0400, Jason Cunliffe wrote:
Please correct me if I misunderstand your idea and its scope..
1. You want to embed meta_data header comments inside Zope files just as Python encourages doc strings
I just want that for objects types which may be ambiguous when uploaded into Zope, i.e. DTML Documents and DTML Methods. For Images for example, this is not necessary because the object type is not ambiguous, and let me explain why: Some program (Zope, Apache, the file command, etc...) scans a few bytes at the beginning of the file to determine it's type (jpeg, tiff, gif, ...), which IMHO is a VERY strong argument to back my point ! Anyway I'm glad someone has seen the light ;-) bye, Jerome Alet
participants (3)
-
Jason Cunliffe -
Jerome Alet -
marc lindahl