[Zope-dev] Re: created z3c.saconfig

[Martijn Faassen]

Hey,

Hermann Himmelbauer wrote:
[snip]
> 1) Why do you need to specify what interface the factory provides, such as 
> here:
> 
> component.provideUtility(engine_factory, provides=IEngineFactory)
> component.provideUtility(utility, provides=IScopedSession)
> 
> Why can't the utilities provide the interface out of the box?

They do, but then registration will only do the right thing if your 
utility only implements a single interface. I also think this makes the 
examples slightly easier to read. Anyway, it's not so important as 
normally registrations would take place from ZCML or using Grok.

> 2) I'd suggest to depict the case where an engine is bound to the session via 
> the "bind=" parameter, as not all of us are that advanced in SA, thus it may 
> be helpful. Moreover, you later on write that "setting up an engine factory 
> is not actually necessary", so the reader may ask himself why the engine 
> utility makes sense at all.

Yes, I wasn't sure about this one. Perhaps I should show this example, 
though it'll expand the example quite a bit and I'm not sure how it's 
helpful. I think we should encourage people to use the utility style 
registration for the engine.

> Perhaps it would be best to sketch the most simple case, with the bind 
> parameter first, then explain what the shortcomings of this case are, and 
> then introduce the engine utility.

Yes, perhaps. I'm not sure whether that's a good idea in a tutorial; one 
that shows examples we don't want to encourage first, or the right 
example right away.

Perhaps this could be done in an extra .txt file where we go into more 
detail about various options.

> 3) I'd suggest to explain the part, where you do a "from z3c.saconfig import 
> Session; session = Session()" a little, and line out that it's used in 
> SQLAlchemy style, e.g.: "After registering the session utility, one can 
> import the Session class vom z3c.saconfig, which offers the same capabilities 
> as a common SQLAlchemy session class."

You're right, that could use some more information. I've expanded the text.

> 4) In the site examples, it reads:
> sm1 = site1.getSiteManager()
> sm1.registerUtility(engine_factory1, provided=IEngineFactory)
> 
> Why is it now "provided" instead of "provides"? Is this a typo or something 
> specific?

It's an annoying inconsistency in the zope.component APIs. It's not a 
typo. Again this is an API that at least Grok hides away for you.

> 5) For the siteScopeFunc part, it would be best if there would already be a 
> generic one in the SiteScopedSession class, although I don't know if this 
> would be possible. However, this would make things simpler for beginners.
> Later on I suggest to explain that it's possible to overwrite this method and 
> what it's for.

I haven't found it easy for z3c.saconfig, as I tried to avoid 
dependencies on things like zope.traversing (which again pull in the 
world), or the ZODB. My intent is for z3c.saconfig to be foundational, 
but that other frameworks will need to fill in some more of the holes. 
My aim is to use this with megrok.rdb, and this will certainly offer a 
Grok-specific way to distinguish between applications.

> The missing bits in this module seem to be:
> 
> 1) Some way to update database parameters, e.g. change your engine: In many 
> web applications, database setup is done by the user during installation 
> (e.g. PHProjekt and many others). The user has some install wizard and inputs 
> the database parameters here, moreover he can change them later on via a web 
> frontend. I think there should be some solution/guideline that aids the 
> programmer in this part. 

I agree that this is still a feature that's missing and should be 
carefully tested.

I'd like to avoid putting knowledge about user interfaces or the exact 
specification of SQLAlchemy parameters in z3c.saconfig though. I'd like 
to offer an infrastructure to reconfigure the engine and then make sure 
the reconfigured engine gets used, but only the minimal one. Again it's 
the task of applications or frameworks that build on top of this to use 
this infrastructure.

> What I can think of is:
> 
> - Simply reregister the engine utility with new parameters

Hm, reregistering a local utility on the fly is rather hard-core, I'd 
like to avoid that and allow modification of engine utilities instead.

> - Provide some "updateEngine" helper function, that queries a site/global 
> registry for an IEngineUtility and alters the database parameters there
> - Somthing like I suggested in my code, where there are specific configuration 
> properties in the utility, that, if changed, recreate the engine.

Yes, I saw the property approach. I didn't want to go for the property 
approach, as I thought if you set 3 properties at once, the engine is 
disposed 3 times.

What I want to offer is an API on IScopedSession and IEngineFactory that 
allows you to replace the engine parameters. This should dispose the 
engine, and then allow for a new engine to be created. I already have a 
sketch of the code on IEngineFactory, taken from your code. That isn't 
enough though, as the scoped session machinery will cache sessions and 
engines indefinitely. We also need an API on IScopedSession therefore 
that can trigger a particular engine (or set of engines) to be throw out 
from the scoped session.

All of this needs careful unit testing. :)

> However, it has to be take special care that when changing the engine, open 
> sessions are not somehow corrupted.

Hm, do you have any details of how this can happen and can be avoided?

> 2) Basically, I can think of 4 main scenarious for a Zope3 + SA integration:
> 
> a) 1 database per Zope3 instance
> b) 1 database per Site
> c) n databases per Zope3 instance
> d) n databases per Zope3 Site
> 
> I suggest to outline that in the beginning of the README.txt along with some 
> introductory words and explain that the setup for these cases differ.

I've added some text to the introduction and adjusted the headings for 
case a) and b).

> (a) and (b) seem to be most common and are covered in z3c.saconfig, but (c) 
> and (d) seem to be missing. 

c) isn't missing, as b) allows a form of c), right? The idea is that if 
you want multiple databases in an instance, you'd set up multiple sites.

d) is indeed not covered. I think it's hard to cover in the current 
design. Laurence Rowe is interested in this use case I believe, and 
perhaps we can find ways to make it possible, but I suspect it might 
have to look quite different and use adapters instead of the Session 
class to get sessions. This because to know which database applies to 
which object you need to know more than just what site you are in, but 
also what object you're dealing with.

We could eventually consider expanding z3c.saconfig with a an 
adapter-driven approach as well to cover this use case.

[snip]
> All in all, thanks a lot for your efforts, this package will be very helpful 
> to others who also need to integrate their application with a RDB!

Thanks again for your helpful code, and thanks for your extensive feedback!

Regards,

Martijn