[AL] As well as an ETA for Btrees, in the absence of ZPatterns documentation, could somebody please post URLs for specific references to further information about Btrees in particular and Zpatterns in general, - eg email list archives and other Wickis particular thread URLs etc. I've been studying the ZPatterns Wicki and code with a view to following up on Kevin's earlier email reference to it which I posted to Zcommerce recently. Didn't notice any explicit reference to Btree folders within ZPatterns. As far as I can make out, Kevin is saying, and appears to be right, that by designing for "Zpatterns" it should be feasible use either ZODB or an RDBM without having to rewrite the whole application, and also feasible to rewrite just the parts that are affected by different facilities on different RDBMs to support more than one RDBM option. (Of course if a Zope application is NOT designed from the start to do that, but simply assumes the ZODB will be the only data source, this would not apply. Unfortunately the two currently available Zope ecommerce products appear to have taken that route). However I got a strong impression ZPatterns may not be as useful as I thought, in a recent exchange with Chris McDonough in an OpenACS bboard (attached, with link to discussion). Any comments and references from Kevin and other people more familiar with ZPatterns would be appreciated. BTW While Btrees might make ZODB usable with very large folders I can't see how it would make it better than using an RDBM in that situation (hopefully under Zope, using ZPatterns). Once you have a lot more data than fits in RAM you are likely to also have a higher ratio of writes to reads than ZODB is "aggressively optimized" for, and need an application design that explicitly optimizes for relational operations on subsets of data rather than navigating from one linked object to another as OODBMs are designed to do. Using an OODBM like ZODB would mean a full disk seek for every item, which RDBMs are designed to avoid. Likewise if you need concurrent external access to the same data via ODBC I don't see how ZODB Btrees would help (eg for ecommerce links to General Ledger and fulfilment etc). What worries me is that in the absence of actual working examples of how to handle RDBMs with ZPatterns, I have had to recommend that someone needing an RDBM based application (Ecommerce) use the OpenACS ecommerce module and just kludge a way to also use Zope for other things in parallel. I'll be discussing the specific implications for ecommerce in Zcommerce, but perhaps here is better for info on ZPatterns and RDBMs generally? Also, although still not sufficiently well informed, I have a strong feeling that the features of Postgresql as an ORDBM as opposed to an ODBM or an RDBM are particularly relevant to Zope. As well as a normal external Python API, Postgresql has the capability to run scripting code as well as C functions inside the database engine as a stored procedure language and also for dynamically loaded user defined data types and comparison operators etc. It is a true "Object" Relational DBMS rather than a conventional RDBMs. Already implemented for Tcl. If some Zope/Python code was put inside it, designed to work with attribute providers and especially indexing agents and rule agents for ZPatterns, it strikes me that the combination could be incredibly powerful. Has there been any discussion of that, or any thoughts now? -----Original Message----- From: jimmie@mail.texoma.net [mailto:jimmie@mail.texoma.net]On Behalf Of Jimmie Houchin Sent: Sunday, June 18, 2000 7:12 AM To: Kevin Dangoor Cc: Zope Mailing List Subject: Re: [Zope] Big folders [JH] Thanks, this is great info. I was planning on developing with ZPatterns but haven't downloaded or began yet. I wasn't aware that a Rack could do that. Cool. I don't mind going to a RDBMS if necessary, but if possible I like sticking with Zope/ZODB. Does anyone have an eta on BTrees for ZODB? Jimmie Houchin Kevin Dangoor wrote:
[snip]
[JH]
My databases will have millions of objects in what would/will be single table/folders. From what I understand ZODB doesn't currently do big folders well.
So I guess my question is, should I continue with the SQL and the RDBMS or is there a solution that I don't know about or understand with ZODB?
[KD]
As Shane mentioned, there's a BTreeFolder in the works. Additionally, you can use the "Rack" from ZPatterns. If you design for a Rack, you can switch to an RDBMS later on without having to change your application code. But, you might not need to switch to an RDBMS, because the current Rack implementation can store objects using BTrees within the ZODB. So, theoretically, you can have great big piles of objects within a single Rack.
Unfortunately, ZPatterns is still very new and there is not a lot of documentation. But, when you start playing with it, things become a bit clearer.
Kevin