[Zope-dev] Re: zope.sqlalchemy, integration ideas

[Laurence Rowe]

Martijn Faassen wrote:
> Laurence Rowe wrote:
>> Martijn Faassen wrote:
> [snip]
>>> Before we talk more about session configuration, please explain why
>>>  we're not talking about engine configuration. :)
>>
>> Engine configuration is a subset of session configuration. You cannot
>>  have a single ScopedSession for a package if you want to have
>> multiple instances of that application.
> 
> I don't understand then what Mike Bayer wrote before:
> 
> Mike Bayer wrote:
>> If you are running different instances each connected to a different
>> engine within one process you wouldn't need any awareness of engines
>> at the object level (therefore no entity_name) and also no engine
>> proxying - you should have separate Session instances for each
>> "process" managed by scoped_session(),  which was designed to handle
>> this.    Multiple apps on one codebase within one process was an
>> original requirement of Pylons as well, though nobody has ever used
>> it.
> 
> That seemed to suggest to me that scoped sessions were an appropriate 
> solution.
> 
> Anyway, back to you:
> 
> [snip]
>  > I'm not sure whether it would be a good idea to wrap this in a session
>  > property, or just register it as an adapter. The only other object that
>  > would need access to the session (either as a property or through
>  > adaption) would be the application instance root object. Something like:
>  >
>  > @adapter(MyApp)
>  > @provider(ISession)
>  > def root_session(context):
>  >     return context._sessioncontext()
> 
> This looks to me like it could be a simple function that looks up a 
> local utility instead:
> 
> def session():
>    return component.getUtility(ISessionContext)()
> 
> We get the right context from the site that way. I don't see the point 
> in trying to re-implement local utilities with adapters while 
> zope.component already does something quite similar for you. That said, 
> I still have hope we can used ScopedSession and forgo a utility lookup 
> here, see below...
> 
>  > And the simplest persistent session context might be:
>  >
>  > class PersistentSessionContext(persistent):
>  >   implements(ISessionContext)
>  >
>  >   def __init__(self, url, twophase=False, engine_kw={}, session_kw={}):
>  >     self.url = url
>  >     self.twophase = twophase
>  >     self.engine_kw = engine_kw
>  >     self.session_kw = session_kw
>  >
>  >   def __call__(self):
>  >     session = getattr(self._v_session, None)
>  >     if session is None:
>  >       engine = getattr(self._v_engine, None)
>  >       if engine is None:
>  >         engine = self._v_engine = create_engine(
>  >           self.url, **self.engine_kw)
>  >       session = self._v_session = create_session(
>  >         bind=engine, twophase=self.twophase, **self.session_kw)
>  >     return session
> 
> Doesn't ScopedSession already take care of this using ScopedRegistry?
 >
> Perhaps a more clever scopefunc is necessary to introduce a per-instance 
> scope? Right now it's only per-thread scope. scopefunc could also do a 
> local utility lookup that gets it a way to uniquely identify the current 
> application (not sure what would be best, object id? zodb id? a unique 
> string generated for each installed application?).
> 
> Something like this:
> 
> def scopefunc():
>    return (component.getUtility(ISessionSiteScope).applicationScope(), 
> thread.getindent())
> 
> Session = scoped_session(sessionmaker(autoflush=True), scopefunc)
> 
>  > A more complex scheme might maintain a dict of ScopedSessions keyed by
>  > application path, outside of the object cache. You could also ensure
>  > that only a single engine is created for a given set of arguments, but
>  > this seems overkill
> 
>  > Everything would then get a session consistently with a call to
>  > ISession(self) or ISession(self.context) in the case of views. No parent
>  > pointers involved.
> 
> I still don't understand why this is nicer than a local utility lookup. 
> I understand the two cooperating applications use case, but that can be 
> easily provided for with using setSite(), just like you *already* need 
> to do to make everything work right with the other local utilities that 
> this application might be using. Above in my scopefunc example I assume 
> that setSite has been set up appropriately.

That would be fine if you had the same configuration across all sessions 
(e.g. they all connected to the same engine / database) or each session 
was configured at the start of every request. Presumably we will want to 
connect different application instances to different databases.

This means that if we have a central register of ScopedSessions, then we 
must key it by some unique application id (path I guess).

def application_session(app):
   try:
     return registry[app.getPath()]()
   except KeyError:
     return registry.setdefault(app.getPath(), 
scoped_session(sessionmaker(**app.getConfiguration())))()

My point about using adapters, or indeed properties to access the 
session, is that the only object needing to access the session which 
cannot look it up directly with Session.object_session(object) is the 
application root object. To me it seems simpler to do this than to 
register utilities. Also it would be nice to have a consistent way to 
lookup the session.

> If you don't use the ZODB at all, you could still set up local sites 
> (I'm not sure how hard this is, but it *should* be possible; 
> zope.component has no knowledge of persistence), or if it's just one app 
> per zope instance, set up a global ISessionSiteScope utility.
>  > We do still need to setup parent pointers though for grok.url and
>  > IAbsoluteURL to work. It looks fairly easy to add location information
>  > to the containers themselves:
> 
> (note that grok.url uses IAbsoluteURL, so we just care about IAbsoluteURL)
> 
> So far this isn't a particular problem in Grok as traversal uses 
> zope.location.located() to locate everything. That said, if you want to 
> directly access an object by accessing ORM-mapped attributes and then 
> get a URL for that object, this won't work. Since it *does* work when 
> you use the ZODB, it'd be nice if it also worked properly with sqlalchemy.
> 
> Hopefully we can indeed make containers behave the right way by making 
> our own MappedCollection.
> 
>  >
>  > from sqlalchemy.orm.collections import MappedCollection, 
> collection_adapter
>  >
>  > class LocatedCollection(MappedCollection):
>  >
>  >   @property
>  >   def __name__(self):
>  >     return collection_adapter(self).attr.key
>  >
>  >   @property
>  >   def __parent__(self):
>  >     return collection_adapter(self).owner_state.obj()
>  >
>  > But locating mapped objects themselves is more complex, they could
>  > exist in more than one collection at once. Perhaps A LocationProxy
>  > could solve this.
> 
> LocationProxy (using zope.location.located()) has worked quite well for 
> me so far.
> 

That sounds promising then.

Laurence