- Zope-Dev - Zope lists

zope.sqlalchemy, integration ideas
by Martijn Faassen 02 Jun '08

02 Jun '08

Hi there, Today I had a discussion with Jasper Spaans about how to go about improving megrok.rdb, Grok's relational database integration which aims to integrate Grok with SQLAlchemy. We started developing megrok.rdb at the Grokkerdam sprint a few weeks ago. We reread the discussion surrounding zope.sqlalchemy for ideas on how to go about integration and configuration. I think these discussions reach wider than just Grok's concerns. Note that I'm not proposing we fold any of these ideas into zope.sqlalchemy itself, which should remain as free of policy as possible; it could become (yet another) extension. Let me run our current thinking by the list. What would be nice in Zope applications (and we think would be good for Grok) would be per-instance database configuration. That is, we want to be able to install multiple instances of the same application and then configure each of them with a different database URN and it should all work, each talking to their own database. Michael Bayer's suggestion involves the use of scoped sessions. He proposed the following code: Session = scoped_session() # start of request engine = get_appropriate_engine() Session(bind=engine) try: # do request finally: Session.remove() Let's go through the steps. First it makes a scoped session object, it then configures it with the right engine at the start of the request (it can do this on a per-class level), and then at the end of the request it removes the Session again, which results in the actual session being closed. Our get_appropriate_engine() would probably look the engine up as a local utility, as Laurence suggested. There is a bit of question about engine configuration, though. If we want to support the use case of looking up the engine URL in a persistent datastore (for instance one URL per location), we have a question of ordering. We cannot do it too early; at the start of the transaction there isn't a ZODB yet to talk to so we can't look up a local utility. We can try doing it just in time: _Session = scoped_session() def Session(*args, **kw): engine = get_appropriate_engine() _Session.bind(bind=engine) return _Session(*args, **kw) Here get_appropriate_engine() could do a component.getUtility() and look up the engine for us, possibly in an application-local way. There's still the question of how this engine got configured in the first place. How does it know the database URL? How does the engine get created after the database URL is known (this might be quite late in the game; it could be stored in the ZODB). It then starts to look more and more attractive to do something similar like collective.lead's IDatabase utility, which can be stored persistently in the ZODB and has a getEngine() method which actually gets the engine (creating it if necessary). If we use sqlalchemy.ext.declarative, we also need to make the declarative extension of SQLALchemy load up the tables at the right point in time. We would also like a way to hook into matters and register some of our own tables and mappers manually. We figured perhaps the utility could fire an event that you can then write a handler for. This way there's less need to subclass the utility just to change some configuration (this is what collective.lead currently requires you to do). If a persistent local utility is in play, it shouldn't fire the configuration event during its own creation, as that would mean it'd only be fired once ever. We want to fire it just after engine creation. I guess the database utility can remain quite simple. Its main tasks would be: * allow access to the engine (creating it the first time) * fire the event for additional configuration when the engine is first created * maintain or somehow obtain the database URL. This could be retrieved from the ZODB if it's a local utility, or it could be hardcoded into a global utility, or it could be retrieved from some config file by a global utility. We could have an expanded variety which also configures things using the SQLAlchemy declarative extension. We still have the question of the 'remove()' bit in Michael's code. We looked at ScopedSession's remove() method, and it looks like it removes the session from the thread-local storage, and it actually closes the session. Closing the session should be taken care of: zope.sqlalchemy's integration with Zope's transaction machinery will close the session. What about the registry cleanup that remove() appears to do? Is this currently being done by zope.sqlalchemy? Should it be? Anyway, a whole lot of abstract talk. I still hope to get some feedback on this. Regards, Martijn

7 26

Zope Tests: 5 OK
by Zope Tests Summarizer 02 Jun '08

02 Jun '08

Summary of messages to the zope-tests list. Period Sun Jun 1 11:00:00 2008 UTC to Mon Jun 2 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Sun Jun 1 21:01:57 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-June/009640.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Sun Jun 1 21:03:28 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-June/009641.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Sun Jun 1 21:04:58 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-June/009642.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Sun Jun 1 21:06:33 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-June/009643.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Sun Jun 1 21:08:03 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-June/009644.html

1 0

Component registry simplification leftovers
by Kapil Thangavelu 02 Jun '08

02 Jun '08

i was setting up a z3 app with a pau utility, and noticed some strange behavior, which tracked down to, the PluggableAuthentication's getPrincipal method, source below. def getPrincipal(self, id): if not id.startswith(self.prefix): next = queryNextUtility(self, IAuthentication) if next is None: raise PrincipalLookupError(id) return next.getPrincipal(id) id = id[len(self.prefix):] for name, authplugin in self.getAuthenticatorPlugins(): info = authplugin.principalInfo(id) if info is None: continue info.credentialsPlugin = None info.authenticatorPlugin = authplugin principal = interfaces.IFoundPrincipalFactory(info)(self) principal.id = self.prefix + info.id return principal next = queryNextUtility(self, IAuthentication) if next is not None: return next.getPrincipal(self.prefix + id) raise PrincipalLookupError(id) i had setup a local site with a pau with a prefix and an ldap auth, lookups for common groups like zope.EveryBody, zope.Authenticated, would in turn call queryNextUtility. however instead of returning the global site manager's principal registry, it would fail in the utility lookup. digging deeper, the queryNextUtility call is passing in a context of the authentication utility itself, which looks like a throwback to the older IComponentLookup adaptation, instead of the current thread local site manager with bases, thats currently used. When the auth utility is passed to queryNextUtility, the default site manager whose bases are queried is the global site manager, resulting in no utility found. Where as passing context none, gives the expected behavior, of looking up the current local site's bases for the auth utility in the global site manager. it appears to me, that this is a throwback/missed refactoring from jim's merge of the component registry refactoring.. can anyone confirm the analysis and the bug? this pattern is present in multiple places in pluggable auth utility implementation, and also, zope.app.security/zope/app/security/vocabulary.py:219 cheers, kapil

1 0

setuptools extension for buildout old-style tar-balls?
by Andreas Jung 01 Jun '08

01 Jun '08

Hi, lots of old-style Zope products are now available as eggs. However a lot of people still don't work with buildout and need the old-style tar-balls. Does anyone know of a setuptools extension for generating and upload the tar-balls directly from a Products.XXXX package? Using the 'sdist' command is not suitable since it includes the 'Products' top-level directory which is not what we want. Andreas -- ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: info(a)zopyx.com - Phone +49 - 7071 - 793376 Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535 Geschäftsführer/Gesellschafter: ZOPYX Limited, Birmingham, UK ------------------------------------------------------------------------ E-Publishing, Python, Zope & Plone development, Consulting

2 1

Zope Tests: 5 OK
by Zope Tests Summarizer 01 Jun '08

01 Jun '08

Summary of messages to the zope-tests list. Period Sat May 31 11:00:00 2008 UTC to Sun Jun 1 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Sat May 31 20:57:13 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009635.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Sat May 31 20:58:43 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009636.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Sat May 31 21:00:14 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009637.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Sat May 31 21:01:44 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009638.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Sat May 31 21:03:15 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009639.html

1 0

Re: SVN: zc.datetimewidget/trunk/ prepare for a sanitizing release, thanks to a wild svn-based release
by Tres Seaver 31 May '08

31 May '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Fred Drake wrote: > On Thu, May 29, 2008 at 12:11 AM, Fred L. Drake, Jr. <fdrake(a)gmail.com> wrote: >> Log message for revision 87005: >> prepare for a sanitizing release, thanks to a wild svn-based release > ... >> CHANGES >> ======= >> >> +0.6.1 (2008-05-29) >> +------------------ >> + >> +Unchanged from 0.5.2, but released with a new version number thanks to a >> +package with an 0.6.1dev-rBFN revision found in the wild. > > This is disgusting. We're going to have to all avoid placing those > nasty releases with Subversion revision numbers in any public > repositories. I'm sure I'm among the guilty, but... we've gotta do > better. Amen. <dead-horse beat="sans-merci"> If we quit keeping "guessed nex version number" on the trunk (or on release branches), such a release would be harder to make. Also, we wouldn't have to make unneeded changes to setup.py on the trunk. </dead-horse> Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver(a)palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD4DBQFIQYqD+gerLs4ltQ4RAvu8AKCXDHKeIJm9cD5HGjOPwPd9SdqctQCXdZXF gTfE3sEsazUJdDJqO48F3A== =Ltvx -----END PGP SIGNATURE-----

2 1

Re: SVN: relstorage/branches/1.1/MANIFEST apparently, setuptools uses SOURCES.txt instead of MANIFEST
by Tres Seaver 31 May '08

31 May '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Shane Hathaway wrote: > Log message for revision 87068: > apparently, setuptools uses SOURCES.txt instead of MANIFEST It generates that file in the .egg-info subdirectory when building an 'sdist' from a VCS checkout (subversion or another with the extension for finding files under version control). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver(a)palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIQYwQ+gerLs4ltQ4RAtjqAKChfjhSZq43UL6zxZzUatdiGln48QCfVFX6 rwibQgcylVfY0ShRVh42gq8= =BitZ -----END PGP SIGNATURE-----

2 1

Zope Tests: 4 OK, 1 Failed
by Zope Tests Summarizer 31 May '08

31 May '08

Summary of messages to the zope-tests list. Period Fri May 30 11:00:00 2008 UTC to Sat May 31 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Test failures ------------- Subject: FAILED (failures=1) : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Fri May 30 21:07:45 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009634.html Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Fri May 30 21:01:29 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009630.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Fri May 30 21:03:04 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009631.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Fri May 30 21:04:40 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009632.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Fri May 30 21:06:15 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-May/009633.html

1 0

Somebody needs to talk at EuropPython.
by Lennart Regebro 30 May '08

30 May '08

So, we need talks at EuropPython about both Repoze and Grok. I can't go, but I want to make sure *somebody* does. :) Hands up: Who is talking about Grok, and who talks about Repoze? :) Other super-cool Zope technologies that should be talked about? -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

4 3

zope 3 component binaries for Python 2.5 on Windows
by Chris Withers 30 May '08

30 May '08

Hey Guys, So, we now have all the Zope 3 components and RestrictedPython working on Python 2.5. Where do I go to get windows binaries installers or similar for these? (ie: the C-components) If they don't exist yet, how do we go about getting them to exist? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

2 2