(resent to here from zope@zope.org due to complete lack of resoonse. Perhaps there should be three levels of mailing list. zope-use, zope-app-dev, zope-dev or something) I'm in the process of of write a zope product that will deal with a lot numerical data (records with a few ids and a few float values). Rightly or wrongly (comments welcome) I've decided to do this inside the ZODB rather than rely on a relation database. This is largely because this data will - not be high-write - I want to do processing on all this of this data on multiple servers - and I am presuming that at some point in the not too distant future zope will let me replicate this data very easily to make it even more highly available. I tried using ZClasses but it seems to run slow and take up space. Instead I'm using Catalog (without ZCatalog) to contain the data. I'm presuming this will be quite efficient (comments welcome). Now to my question, How do Catalog plugable brains work? I've looked but can't see what the brain class has to look like to work. My first attempts don't seem to work. The classes are created but don't contain the data. Is the record data passed into the constructor? Also in my searches I came across lots of references to something called ZTables. This seems to be a Catalog with a UI that is about lots of tabular information (rather than a ZCatalog which is specialized to replicating and indexing existing objects). Is this dead? If not where is it? If so, why? It seems like a really good idea to me. It seems to be there are times when objects (esp ZClasses) are too heavy? Anyone with comments about how ZPatterns fits into all of this would also be welcome.
Jay, Dylan wrote:
(resent to here from zope@zope.org due to complete lack of resoonse. Perhaps there should be three levels of mailing list. zope-use, zope-app-dev, zope-dev or something)
I'm in the process of of write a zope product that will deal with a lot numerical data (records with a few ids and a few float values). Rightly or wrongly (comments welcome) I've decided to do this inside the ZODB rather than rely on a relation database. This is largely because this data will
- not be high-write - I want to do processing on all this of this data on multiple servers - and I am presuming that at some point in the not too distant future zope will let me replicate this data very easily to make it even more highly available.
Write a Python Product. See the Zope Developers Guide for details. http://www.zope.org/Documentation/ZDG/Products.dtml
Anyone with comments about how ZPatterns fits into all of this would also be welcome.
I can't see that it would particularly help you. -- Steve Alexander Software Engineer Cat-Box limited
Jay, Dylan wrote: [snip]
Also in my searches I came across lots of references to something called ZTables. This seems to be a Catalog with a UI that is about lots of tabular information (rather than a ZCatalog which is specialized to replicating and indexing existing objects). Is this dead?
I got the impression that it is. It's probably not been updated for a long time; I think not even to Zope 2, but I may be mistaken.
If not where is it?
It was never released as open source, so it isn't floating around except probably at Zope Corporation somewhere.
If so, why? It seems like a really good idea to me. It seems to be there are times when objects (esp ZClasses) are too heavy?
Obviously ZClasses are heavy, but tabular data in Python goes in some kind of object anyway, so it's rather hard to avoid *objects*, unless you use some external database. MetaKit has a nice Python interface and is lightweight, for instance. I haven't heard of anyone integrating it with Zope yet, though: http://www.equi4.com/metakit Anyway, Python objects are lightweight enough for most purposes. I'm not entirely sure why the idea of ZTables went away so completely. Python tables in the ZODB combined with the catalog should make for an interesting system to play with. Though perhaps there are products out there that actually do this. Come to think of it, MetaPublisher (not to be confused with MetaKit) can use a ZODB backend, can't it? Regards, Martijn
I'm not entirely sure why the idea of ZTables went away so completely. Python tables in the ZODB combined with the catalog should make for an interesting system to play with. Though perhaps there are products out there that actually do this. Come to think of it, MetaPublisher (not to be confused with MetaKit) can use a ZODB backend, can't it?
Stephan Richter just told me that the ZOQL (short for "use SQL-like syntax within the ZODB") has become a reality. Maybe that could be interesting for your problem domain, too.
Stephan Richter just told me that the ZOQL (short for "use SQL-like syntax within the ZODB") has become a reality. Maybe that could be interesting for your problem domain, too.
I did want to announce it officially yet (and I won't), but the URL is: http://demo.iuveno-net.de/iuveno/Products/ZOQLMethod Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development & Technical Project Management
Martijn Faassen wrote:
I'm not entirely sure why the idea of ZTables went away so completely. Python tables in the ZODB combined with the catalog should make for an interesting system to play with. Though perhaps there are products out there that actually do this. Come to think of it, MetaPublisher (not to be confused with MetaKit) can use a ZODB backend, can't it?
ZTables was a Zope1 product. We never updated it for Zope2. There's so much new machinery in Zope, particularly regarding indexing, that ZTables would really only serve as interesting requirements input. I certainly think that some kind of generic table tool in Zope would be useful. I'm sure that others could do a better job of it than ZTables did, in light of all that's happened in Zope and the CMF since then. In fact, Formulator probably has a role as well. --Paul
On Thursday 04 October 2001 07:34 pm, Jay, Dylan allegedly wrote:
I tried using ZClasses but it seems to run slow and take up space. Instead I'm using Catalog (without ZCatalog) to contain the data. I'm presuming this will be quite efficient (comments welcome).
Yup that should be pretty efficient. Plus you can index and query the thing fairly easily.
Now to my question, How do Catalog plugable brains work? I've looked but can't see what the brain class has to look like to work. My first attempts don't seem to work. The classes are created but don't contain the data. Is the record data passed into the constructor?
I'm not an expert on brains per se, but my understanding is that they allow you to wrap functionality from a particular class around data stored in a table (such as a Catalog or an external database) without making each record an instance of the class. This is how, when the Catalog returns data, you get the getURL and getObject methods for each record (among others). TinyTable and Z SQL Methods also support this functionality, and it is exposed in the management interface. This would allow you to store the actual data as regular metadata elements in the Catalog, and get objects out when you query it. It would avoid storing the data twice, once in the objects and again in the metadata and all of the disadvantages this causes in terms of storage and synchronization.
Also in my searches I came across lots of references to something called ZTables. This seems to be a Catalog with a UI that is about lots of tabular information (rather than a ZCatalog which is specialized to replicating and indexing existing objects). Is this dead? If not where is it? If so, why?
AFAIK, Catalog contains the only remaining remnants of ZTables in Zope.
It seems like a really good idea to me. It seems to be there are times when objects (esp ZClasses) are too heavy?
ZClasses impose a performance penalty due to their inherent complexity and overhead. You would do much better creating a straight Python class for applications where performance and overhead per record is an issue. I personally avoid using ZClasses for everything but the simplest custom objects.
Anyone with comments about how ZPatterns fits into all of this would also be welcome.
My understanding is that ZPatterns is for abstracting the application from the storage. It is most beneficial for creating storage-independent applications. It would benefit you if that is a requirement, no so much if not. An example might be an application with data in objects, catalogs and external databases. ZPatterns would let you create an abstraction layer so that the application would not need to be aware of where any given piece of data was coming from. You could therefore change your data storage later and ideally you would only have to change the abstraction layer, not the application itself. /---------------------------------------------------\ Casey Duncan, Sr. Web Developer National Legal Aid and Defender Association c.duncan@nlada.org \---------------------------------------------------/
participants (7)
-
Casey Duncan -
Jay, Dylan -
Joachim Werner -
Martijn Faassen -
Paul Everitt -
Stephan Richter -
Steve Alexander