I've been working on a new web framework named (provisionally) repoze.bfg. The relevant README is here: http://svn.repoze.org/repoze.bfg/trunk/README.txt It is essentially a "Zope lite". It is Zope-ish in that: - it uses a lot of Zope libraries and concepts (interfaces, some CA constructs, ZCML) - it uses graph-traversal-based URL dispatch. - by default it provides ZPT (actually z3c.pt) templates. - it provides built-in authorization facilities based on permissions and users/groups. It is unlike Zope (variously 2 or 3) inasmuch as: - It uses a different publisher and authorization model. - It is heavily dependent on WSGI. - Its equivalent of the publisher doesn't attempt to traverse into *code* stored in the object graph. All code is on the filesystem. The only thing in the object graph is "model" data. - It doesn't attempt to provide any server functionality. This duty is left over to WSGI. - It doesn't provide *authentication* functionality (just authorization functionality). Authentication is left up to upstream server/middleware (e.g. repoze.who). - It's not meant to be "plugged in to". A repoze.bfg application is not a plugin to repoze.bfg (e.g. it's not a Zope2 Product). Instead, a repoze.bfg application is just a Python package that happens to use repoze.bfg facilities. - The graph root is not effectively assumed to be a ZODB database. - It provides no built-in through-the-web management interface. It's unlike Grok inasmuch as: - It doesn't try to hide ZCML for configuration. If I were to characterize it succintly and informally, I migh say it's a bit like Django built around graph traversal dispatch instead of URL-based dispatch. Speed is a priority. A "hello world" template is currently clocking in at around 550 req/sec on my MacBook (including security lookups). It's my intent to continue to develop it, along with a package named "repoze.lemonade", which will integrate repoze.bfg with ZODB and provide various persistence-related features (folders, object events, indexing and so forth). In any case, I thought maybe some folks would be interested in using/contributing to it. It can be checked out via: svn co http://svn.repoze.org/repoze.bfg/trunk/ Thanks, - C
Chris McDonough wrote:
I've been working on a new web framework named (provisionally) repoze.bfg.
This looks very interesting; I'd be curious to see if this could be useful for Vudo. I'd like it very much if Vudo could sit on top of a more general framework (not just the Zope 3 libraries). Early on, the idea was that this could be Grok, but it quickly turned out that Grok makes too many assumptions for our use. I recently pasted a basic Pylons application and it gives you something that I think would be attractive in a Zope/Vudo/Bfg-based setup: A canonical directory structure, e.g. ./templates ./routers ./config etc. (can't remember the details) Perhaps this could benefit the following scenario: -- Set me up with a new Zope/Vudo/Bfg-application and give me some starting points. If Bfg can provide the lower layer, then I think Vudo will be great for providing the higher layers, e.g. -- skinning -- content types and widgets -- authoring -- admin \malthe
Previously Malthe Borch wrote:
Chris McDonough wrote:
I've been working on a new web framework named (provisionally) repoze.bfg.
This looks very interesting; I'd be curious to see if this could be useful for Vudo. I'd like it very much if Vudo could sit on top of a more general framework (not just the Zope 3 libraries).
Early on, the idea was that this could be Grok, but it quickly turned out that Grok makes too many assumptions for our use.
I recently pasted a basic Pylons application and it gives you something that I think would be attractive in a Zope/Vudo/Bfg-based setup: A canonical directory structure, e.g.
./templates ./routers ./config
templates - page templates controllers - the pylons-version of views. sort-of. config - everything related to the appplication configuration lib - all our utilities models - data models public - static web content Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
Malthe Borch wrote:
Chris McDonough wrote:
I've been working on a new web framework named (provisionally) repoze.bfg.
This looks very interesting; I'd be curious to see if this could be useful for Vudo. I'd like it very much if Vudo could sit on top of a more general framework (not just the Zope 3 libraries).
Early on, the idea was that this could be Grok, but it quickly turned out that Grok makes too many assumptions for our use.
I recently pasted a basic Pylons application and it gives you something that I think would be attractive in a Zope/Vudo/Bfg-based setup: A canonical directory structure, e.g.
./templates ./routers ./config
etc. (can't remember the details)
Sure. I think one of us (maybe Paul?) signed up to write a PasteScript template to lay out a directory structure something like this. We don't currently provide an easy way to serve out a static directory full of content. We'd need to add that (or decide not to add it in favor of letting a separate "resource" server serve the static stuff).
Perhaps this could benefit the following scenario:
-- Set me up with a new Zope/Vudo/Bfg-application and give me some starting points.
If Bfg can provide the lower layer, then I think Vudo will be great for providing the higher layers, e.g.
-- skinning -- content types and widgets -- authoring -- admin
Sounds good to me. The plans are to keep BFG mostly policy-agnostic save for the very basics (graph traversal, a default templating language, and a calling and response convention for views). I had planned to create another package named repoze.lemonade which: - Wired in ZODB - Defined base classes for folderish and leafish types. - Had an object add/remove/move event model. - Did indexing of content. It sounds like Vudo could either build on top of that or just *be* that. It might be better to continue layering stuff, I suppose, without going "straight to the content management layer". Would it be appropriate for Vudo to build on something like that? What would Vudo need out of a framework? - C
Chris McDonough wrote:
I had planned to create another package named repoze.lemonade which: ... - Did indexing of content.
What were you thinking of for indexing? Just catalog stuff? More general? There's been a tension in the opencore stuff with the catalog, mostly that it's easy to setup and use for things, but it doesn't really work for things outside of the ZODB. Or, I guess theoretically you could catalog things not in the ZODB, but it's never happened. That said, there's a real need for cross-system indexing of different kinds. There's text search indexes, but other kinds of more strict indexes also make sense. It would be very handy to have an index that wasn't tied to the ZODB, a database, or anything else. (It could be implemented using the ZODB or a database, of course.) -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
Ian Bicking wrote:
Chris McDonough wrote:
I had planned to create another package named repoze.lemonade which: ... - Did indexing of content.
What were you thinking of for indexing? Just catalog stuff? More general?
I was considering just using zope.index but I haven't really thought much about it yet.
There's been a tension in the opencore stuff with the catalog, mostly that it's easy to setup and use for things, but it doesn't really work for things outside of the ZODB. Or, I guess theoretically you could catalog things not in the ZODB, but it's never happened.
IMO, mostly it's a matter of deciding what "index" and "catalog" means for searching. If you only need full-text search, some indexing systems are totally inappropriate. Likewise, if you only need "field" indexes that match just one particular value, it's sometimes vastly simpler just to come up with your own system based on, e.g. a relational table, than it is to try to use somebody else's "indexer" or "catalog".
That said, there's a real need for cross-system indexing of different kinds. There's text search indexes, but other kinds of more strict indexes also make sense. It would be very handy to have an index that wasn't tied to the ZODB, a database, or anything else. (It could be implemented using the ZODB or a database, of course.)
Xapian seems like a good choice too. It has its own persistence mechanism and reasonable Python bindings (although from experience maybe slightly memory-leaky). - C
Chris McDonough wrote:
There's been a tension in the opencore stuff with the catalog, mostly that it's easy to setup and use for things, but it doesn't really work for things outside of the ZODB. Or, I guess theoretically you could catalog things not in the ZODB, but it's never happened.
IMO, mostly it's a matter of deciding what "index" and "catalog" means for searching. If you only need full-text search, some indexing systems are totally inappropriate. Likewise, if you only need "field" indexes that match just one particular value, it's sometimes vastly simpler just to come up with your own system based on, e.g. a relational table, than it is to try to use somebody else's "indexer" or "catalog".
They are similar in that they both need to get information about updates, and a way to reindex. Full text search can be a little lazier, as being 100% up-to-date isn't such a big issue.
That said, there's a real need for cross-system indexing of different kinds. There's text search indexes, but other kinds of more strict indexes also make sense. It would be very handy to have an index that wasn't tied to the ZODB, a database, or anything else. (It could be implemented using the ZODB or a database, of course.)
Xapian seems like a good choice too. It has its own persistence mechanism and reasonable Python bindings (although from experience maybe slightly memory-leaky).
Yes, once you get the documents into it. Actually, it's really a many-to-many notification system that's needed. We have one that needs documenting and review (http://www.openplans.org/projects/cabochon/project-home) but while it handles notifications and events (as do several other systems), that doesn't cover reindexing the site. And then you also need a set of useful endpoints, but those can grow over time. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
Ian Bicking wrote:
Chris McDonough wrote:
There's been a tension in the opencore stuff with the catalog, mostly that it's easy to setup and use for things, but it doesn't really work for things outside of the ZODB. Or, I guess theoretically you could catalog things not in the ZODB, but it's never happened.
IMO, mostly it's a matter of deciding what "index" and "catalog" means for searching. If you only need full-text search, some indexing systems are totally inappropriate. Likewise, if you only need "field" indexes that match just one particular value, it's sometimes vastly simpler just to come up with your own system based on, e.g. a relational table, than it is to try to use somebody else's "indexer" or "catalog".
They are similar in that they both need to get information about updates, and a way to reindex. Full text search can be a little lazier, as being 100% up-to-date isn't such a big issue.
I'd typically handle the "info about updates" bit by sending an event (which I suspect cabochon is all about).
That said, there's a real need for cross-system indexing of different kinds. There's text search indexes, but other kinds of more strict indexes also make sense. It would be very handy to have an index that wasn't tied to the ZODB, a database, or anything else. (It could be implemented using the ZODB or a database, of course.)
Xapian seems like a good choice too. It has its own persistence mechanism and reasonable Python bindings (although from experience maybe slightly memory-leaky).
Yes, once you get the documents into it. Actually, it's really a many-to-many notification system that's needed. We have one that needs documenting and review (http://www.openplans.org/projects/cabochon/project-home) but while it handles notifications and events (as do several other systems), that doesn't cover reindexing the site. And then you also need a set of useful endpoints, but those can grow over time.
Stuff that I'm not sure about above: "many-to-many notification system" "reindexing the site" That said... I guess what you're saying is that it's hard to send an event that makes it clear what needs to be done as a result of the event being sent. In the case where you have a content node that holds data, that node would be attched to the event. it's pretty clear that you need to do.. that thing changed, and the indexing code needs to inspect it and reindex whatever indexes you've got sitting around that are willing to index data about those types of nodes. In the case where you need to deal with the simultaneous case that some row in an RDB was changed, it's not quite as clear. What does it mean and who should do what with that event? It's all policy at that point. I'm not sure any framework helps. It's just block-and-tackle event reception and reaction, "do the needful", isn't it? - C
Malthe Borch wrote:
Chris McDonough wrote:
I've been working on a new web framework named (provisionally) repoze.bfg.
This looks very interesting; I'd be curious to see if this could be useful for Vudo. I'd like it very much if Vudo could sit on top of a more general framework (not just the Zope 3 libraries).
Early on, the idea was that this could be Grok, but it quickly turned out that Grok makes too many assumptions for our use.
You're using Zope 3, right? Zope 3 makes the same set of assumptions Grok does, with very minor differences indeed. Could you be more explicit about what exactly in Grok was making too many assumptions? Wasn't it simply a case of different tastes, instead of assumptions that get in the way? Regards, Martijn
Martijn Faassen wrote:
Could you be more explicit about what exactly in Grok was making too many assumptions?
First a word on terminology: I mean Grok, the framework, not the declarative extensions to the component architecture (which I simply haven't gotten to yet). We felt that Grok was too much about object publishing; there's a model and a view for that model (and that view has a template). This didn't fit so well into our approach. On Vudo, the view is always a "layout". It's then up to the layout to provide regions in which you can plug in a "region content provider". Typically, you'd plug in at least the following: -- Title provider: Renders the title of the page (in <title></title>) -- Content provider: Renders the content of the page as widgets There are many more options to this scheme. You could plug in a portlet manager, or a viewlet manager or a global navigation. Next, we didn't want to use ZODB, because we wanted to try something completely different. So now we've written Dobbin which pretty much emulates ZODB, but on a SQL storage (so much for trying something new). I like Grok and I think it's great for writing Zope *applications*; but we didn't find it such a good match for Vudo. I still want to try grokcore.component because there are some obvious candidates for declarative component setup in a system like ours (content-types, widgets, forms, etc.). \malthe
Malthe Borch <mborch@gmail.com> writes:
On Vudo, the view is always a "layout". It's then up to the layout to provide regions in which you can plug in a "region content provider". Typically, you'd plug in at least the following:
-- Title provider: Renders the title of the page (in <title></title>) -- Content provider: Renders the content of the page as widgets
There are many more options to this scheme. You could plug in a portlet manager, or a viewlet manager or a global navigation.
Someone mentioned Vudo to me last week. I said that I was tempted to write an extension to Grok that lets you create a "grok.Layout" template that maybe is inherited throughout your whole site if you don't specify otherwise, and then a "grok.View" on an object would just generate content for the "main" pane inside of that layout, and viewlets could keep working like they do today but fill in other parts of the layout. This is kind of how people look to be doing things today with the viewlets revolution that's going on in Grok land; look at: http://svn.zope.org/Grokstar/trunk/src/grokstar/blog.py?rev=87483&view=auto and note how many of the "grok.View" objects are specifying the same dratted template, over and over again, and then rely on viewlets to actually fill in the varying parts of each page that change with the content. It struck me as a situation that is just crying out for a "grok.Layout" local utility that provides the template everywhere under a part of a site automatically, so that the "grok.template(...)" declaration does not have to get repeated so incessantly. It really looks to me like Grok best-practices are evolving towards a layout-centric, rather than a macro-centric, approach, and that something like the Vudo approach would make this all easier to manage. Do I sound on-target, or like I'm missing something? -- Brandon Craig Rhodes brandon@rhodesmill.org http://rhodesmill.org/brandon
[taking some cc-s out, taking it also to grok-dev] Brandon Craig Rhodes wrote: [snip]
Do I sound on-target, or like I'm missing something?
I think there is definite room for improvement. Before you write anything new though, you should explore z3c.pagelet. We started discussing integrating this approach into Grok, also at the Grokkerdam sprint. There are a few open issues concerning z3c.pagelet integration: * we need to make sure that it becomes dirt-simple to configure using Grok. * we need to make this work smoothly with grok.View somehow; ideally grok.View will become pagelets without breaking backwards compatibility. * I for one need to internalize the details. I got it when Christian Theune explained it at the Grokkerdam sprint, but now things are fuzzy again. :) Vudo is interesting but fairly experimental at this stage, while z3c.pagelet have been used in production at various locations, so I think this is a better bet at early integration into Grok. That's not to stop further explorations on Vudo's part - I'm sure there is room for improvement beyond pagelets. Regards, Martijn
Brandon Craig Rhodes wrote:
Someone mentioned Vudo to me last week. I said that I was tempted to write an extension to Grok that lets you create a "grok.Layout" template that maybe is inherited throughout your whole site if you don't specify otherwise, and then a "grok.View" on an object would just generate content for the "main" pane inside of that layout, and viewlets could keep working like they do today but fill in other parts of the layout.
Right, that's pretty much it. I should note that our layout is essentially: -- a classic HTML document -- whatever css and js it may reference -- a set of mappings from region definitions to xpath expressions
This is kind of how people look to be doing things today with the viewlets revolution that's going on in Grok land; look at:
http://svn.zope.org/Grokstar/trunk/src/grokstar/blog.py?rev=87483&view=auto
and note how many of the "grok.View" objects are specifying the same dratted template, over and over again, and then rely on viewlets to actually fill in the varying parts of each page that change with the content. It struck me as a situation that is just crying out for a "grok.Layout" local utility that provides the template everywhere under a part of a site automatically, so that the "grok.template(...)" declaration does not have to get repeated so incessantly.
Interesting approach. While I can see the benefit of being able to plug these viewlets in and out, it may be YAGNI. If you really wanted users to be able to plug, say, search *out*, how'd they do it? Perhaps portlets are better suited here. With Vudo we really wanted to make sure that whatever the user might want to order around or otherwise manage, it was possible. Viewlets are *great* for things like status-messages, document actions and other such helpers, but I'm not sure I like it as a means to compose pages.
It really looks to me like Grok best-practices are evolving towards a layout-centric, rather than a macro-centric, approach, and that something like the Vudo approach would make this all easier to manage.
Yes, I'm very sure now that layouts are right and macros are wrong. It's an easy pattern to understand and it works well with designers. \malthe
Hi there, On Thu, Jul 17, 2008 at 9:19 PM, Malthe Borch <mborch@gmail.com> wrote:
Martijn Faassen wrote:
Could you be more explicit about what exactly in Grok was making too many assumptions?
First a word on terminology: I mean Grok, the framework, not the declarative extensions to the component architecture (which I simply haven't gotten to yet).
We felt that Grok was too much about object publishing; there's a model and a view for that model (and that view has a template). This didn't fit so well into our approach.
[snip explanations] Thanks for those.
I like Grok and I think it's great for writing Zope *applications*; but we didn't find it such a good match for Vudo. I still want to try grokcore.component because there are some obvious candidates for declarative component setup in a system like ours (content-types, widgets, forms, etc.).
So basically you felt Zope 3 wasn't a good match for Vudo, in the sense that the normal browser:page wasn't really want you wanted either, right? Similar to the way you could extend Zope 3 with your own new ZCML directives to set up the way you'd like views to work (I'm not sure whether you're doing this), you could as well extend Grok with your own new grokkers. I just didn't want Grok singled out here - I don't to leave people with the impression that Grok locks them into a certain approach any more than Zope 3-the-application-framework does; I don't believe it does. Of course Zope 3-the-libraries leave the options more open, witness this thread. The various grok libraries (martian, grokcore.component) should be seen as part of this wider ecosystem as well. I hope that some of your explorations concerning layouts could be plugged into Grok as a library eventually. Regards, Martijn
Martijn Faassen wrote:
So basically you felt Zope 3 wasn't a good match for Vudo, in the sense that the normal browser:page wasn't really want you wanted either, right? Similar to the way you could extend Zope 3 with your own new ZCML directives to set up the way you'd like views to work (I'm not sure whether you're doing this), you could as well extend Grok with your own new grokkers.
Correct. And we did create a new directive to define layouts. I actually think ZCML is very suitable when you're configuring stuff that hasn't to do with code. Whether it's good for configuring code is for another discussion.
I just didn't want Grok singled out here - I don't to leave people with the impression that Grok locks them into a certain approach any more than Zope 3-the-application-framework does; I don't believe it does. Of course Zope 3-the-libraries leave the options more open, witness this thread. The various grok libraries (martian, grokcore.component) should be seen as part of this wider ecosystem as well.
That's a good point. Certainly Grok proves to be quite flexible, but it's still very centralized on defining a set of components and have them automatically glued together. I think Grokkish ideas will make much sense in Vudo *applications*. For the framework code I'm a bit more conversative.
I hope that some of your explorations concerning layouts could be plugged into Grok as a library eventually.
The layout stuff is definitely easy to reuse and I think Brandon already has a good grasp on how it might fit in. The key idea with ``vudo.layout`` is that you can just plug in HTML and have it work. \malthe
Hey, Chris McDonough wrote: [snip]
It's unlike Grok inasmuch as:
- It doesn't try to hide ZCML for configuration.
Hiding is the wrong word. Grok doesn't hide ZCML for configuration. It simply replaces ZCML with a different configuration mechanism that I for one think is an improvement. One that you could easily start using with grokcore.component, I may add. You can mix and match with ZCML-based configuration as you please. That brings me to another difference with Grok: your framework is also unlike Grok inasmuch it's incompatible with Zope 3, correct? Regards, Martijn
Martijn Faassen wrote:
Hey,
Chris McDonough wrote: [snip]
It's unlike Grok inasmuch as:
- It doesn't try to hide ZCML for configuration.
Hiding is the wrong word. Grok doesn't hide ZCML for configuration. It simply replaces ZCML with a different configuration mechanism that I for one think is an improvement. One that you could easily start using with grokcore.component, I may add. You can mix and match with ZCML-based configuration as you please.
Thanks for the clarification.
That brings me to another difference with Grok: your framework is also unlike Grok inasmuch it's incompatible with Zope 3, correct?
Correct, if you're talking about using the Z3 publisher, although repoze.bfg is based on zope.component and zope.interface internally. - C
Hey, Chris McDonough wrote: [snip]
Correct, if you're talking about using the Z3 publisher, although repoze.bfg is based on zope.component and zope.interface internally.
I wasn't primarily thinking about the publisher, more about such things like existing utilities, events, existing content types and so on. Regards, Martijn
Martijn Faassen wrote:
Hey,
Chris McDonough wrote: [snip]
Correct, if you're talking about using the Z3 publisher, although repoze.bfg is based on zope.component and zope.interface internally.
I wasn't primarily thinking about the publisher, more about such things like existing utilities, events, existing content types and so on.
Most of those would work. We don't implement local site managers, but we do use a normal component registry, so if you've got a global utility that you'd like to use, you'd just put its registration into the configure.zcml, make sure you had the right package on the filesystem to handle that registration. ... Speaking of site managers, if you'd like to be able to use more than one separtely-configured grok application in a process (given that you can't right now because the global registry is a singleton), you might want to steal this hack. I myself stole the idea from Stephan's z3c.baseregistry: http://svn.repoze.org/repoze.bfg/trunk/repoze/bfg/registry.py Sending events would of course work. We don't have any event *listeners*, of course, so it would be pointless to send, e.g. object events. That said, "contenty" stuff is not really the job of repoze.bfg, but of a higher-layer framework. repoze.bfg is about traversing the content graph and publishing views and templates. Everything else is the domain of the "application". Existing content types would work to the extent that you'd just use them as data in the ZODB, allowing bfg to traverse the ZODB as the root. This is also the domain of the "application" in terms of bfg. - C
participants (6)
-
Brandon Craig Rhodes -
Chris McDonough -
Ian Bicking -
Malthe Borch -
Martijn Faassen -
Wichert Akkerman