On 22 Nov 1999 19:40:00 -0600, Robert Leftwich wrote:
Robert Leftwich wrote:
I have a requirement to be able to apply a user-definable filter to all the views on the object database. Obviously this would be best applied at a low a level as possible in order to minimise the impact on the application code and to be made as generic as possible for maximum flexibility. To save re-inventing the wheel, has anyone else had this requirement and if so how did you implement it ?
I'm off in a little different area which is an extension of the TinyTables concept where I'm wrapping information obtained from several different locations into an object which is then unrolled or 'SubMined' as Ty calls it. :^) The information comes from the filesystem, the Zope database, and hopefully soon, from other database queries, etc. Right now I roll almost all the results into ZRDB.Results objects so I can more consistently filter them and carry the filter results along as the user traverses through the data. This has a chance of allowing me to 'join' my ZRDB.Results with other database results, etc. I've tried to separate functionality to provide for more flexibility with selectable filters that are applied to the SubMine results object via __bobo_traverse__, in addition to DTML-based rendering filters which are not 'carried around' as the user traverses the objects. The SubMine filters are listed in order and can be selectively removed from the 'filter list'. This concept was derived from the 'Datamining' product posted earlier this year, along with some initial work that Ty helped me with. The problem is that the __bobo_traverse__ mechanism I'm using is fraught with many problems but I can't see any way around them right now. This is somewhat applicable to the discussion on schemas because these objects change on a regular basis (SubMines on every request), but storing *some* of those changes as transactions in the Zope database is simply not appropriate. Right now, it appears to me that Zope isn't well suited to dealing with persistent, user-defined volatile filters, schemas, etc. It really wants to pass those things off to database-like queries. For a while I thought that I ought to do just that with my datamining stuff - just write a separate server that handles the things that Zope doesn't deal very well with, but the problem is that Zope is so darned close to what *is* needed that it'd be pretty stupid to go off and write something else. I'd rather see if there is a way to convince Zope to differentiate between objects that need transacted in the Zope database and objects that just need to be cached in the Zope database, otherwise retaining the same behavior.
Phillip J. Eby wrote :
Ty Sarna and I have come up with a concept for this, which we have not yet implemented, as we are still finalizing design details. We believe that it could be made quite general. The basic idea is that you have a
(Snip some excellent design/use case discussion [not to mention the odd head spin :-) ]...)
I have been thinking about this on and off for the last week or so and I wanted to bounce around an idea or two.
Given that one of the main criteria (for me, at least) is the ability to apply different filters on a per user basis, I am contemplating adding a filter keyword to the in and tree tags. There already exists a very specialised filter in both of these tags (the skip_unauthorized flag - which is a filter for unauthorised objects), so it should be easy to enhance this to support a more generic filter of the type proposed by Philip and Ty.
The factors pushing me toward this solution are that filtering is generally a UI-driven requirement and I felt very uncomfortable about retrieving user-selected filters from the REQUEST variable at low levels in the code (I was experimenting with filtering in the objectId() method of ObjectManager). Not to mention that this solution only worked for ObjectManager associated data and fails when you have lists or dictionaries or external data access such as SQL.
Anyone have any thoughts or comments ?
The SubMine filters that Ty and I worked on attempted to mimic SQL query commands as it provided a familiar environment. If we are really talking about general-purpose filters that can operate on ZRDB.Results objects (in particular), I'd sure like to see what could be done here. BTW, Phillip kinda lost me. :^) Ty - You know what I'm working on. Are we talking about anything remotely related? :^) I'm ready to ditch what I'm doing if there's a better way out there. ---- FWIW, MS and MIT are offering some funds for developing Open Source educational resources that I think we stand a chance of taking advantage of in this area... (http://swissnet.ai.mit.edu/projects/i-campus/) "I-Campus is a collaborative initiative of MIT and Microsoft Research to conduct research and create new technologies that will set the pace for university education in the next five to ten years. The project aims at fostering excellence in technology-enhanced education, through producing materials that adhere to open standards, with results and source code which can be widely published and disseminated. I-Campus was launched in October 1999 as a five-year research partnership between MIT and Microsoft, but we plan to engage additional academic and industry partners as the project evolves." I am in touch with some people who are considering a proposal that some of these issues might address. A few application-specific things would need to be developed but I think we are headed down similar paths and ought to consider working together. Any interested parties? Paul and the rest of you DC folks, there are a number of issues with the I-Campus research that are darned close to what the AISD proposal that we worked on eventually wanted to address. You might want to take a look and see if there are partnership opportunities available to pick up where we left off. Kent