[Grok-dev] Re: STORM howto

[Laurence Rowe]

Martijn Faassen wrote:
> Hey Laurence,
> 
> Thanks for participating in this discussion!
> 
> Laurence Rowe wrote:
>> (wasn't sure what list I was being cc'ed into ;-)
> 
> [snip]
>> collective.mercury takes the `transmutation` part of Alchemist for
>> creating zope schemas from sqlalchemy metadata and writes out the
>> schemas,]
> 
> You do generate the schemas in memory and then write them out? What 
> prompted the decision to write them out?

So I could see them and then more easily know how to subclass them - for 
instance I needed to change the types of some fields (e.g. choice 
instead of int, validators). Actually, recently I've moved away a bit 
from the whole SQL first approach, mainly because I've changed database 
twice in the past month.

Starting now I would probably hack around with the declaritive stuff to 
work out how to annotate the field info to the Column definitions and 
autogenerate if not supplied.

> [snip]
> 
> [collective.lead]
>> strictly zodb3 is not required, only transaction. The elro-tpc branch
>> will become 2.0 fairly soon. It pulls together some work done by
>> bitranch using scoped sessions and supporting the MyClass.query magic
>> and automatically associating new instances with sessions as well as
>> adding twophase and savepoint support. I just need to do a bit of
>> tidying up before I merge to trunk.
> 
> I'm not sure what features you're referring to here; I'm not that 
> familiar with SQLAlchemy. Are scoped sessions and MyClass.query magic 
> both SQLAlchemy features? Would you recommend a new project starts with 
> this branch?

ScopedSessions provide per thread sessions and are a good thing. 
Randomly they also seem to provide the option (using a special 
session.mapper) to annotate your original class with information about 
which metadata/session they belong too. Thinking again, I don't 
recommend you use them. Too much magic. I've moved it to an experimental 
module for now.

The more I think of it, I would limit the IDatabase interface to 
session, engine and connection. It seems that the declarative approach 
is to keep metadata separate. This should allow for having multiple 
sites with their own databases using the same code for mappers.


> Could you explain how collective.lead is better than z3c.sqlalchemy and 
> z3c.zalchemy? :)

It does less. I found it easier to understand and therefore extend to 
add twophase/savepoint support.

tx.py is now completely self contained. So you can use that without the 
IDatabase stuff if you wish.


> [snip]
>> Also, take a look at the new declaritive extension in SA 0.4.4. It
>> looks kind of cool, especially for the grok approach.
> 
> Thanks for the hint. Is this part of SA 0.4.4?

Yes, see: http://www.sqlalchemy.org/docs/04/plugins.html#plugins_declarative

> 
>> It'd be really nice to see collective.tin work with grok. I don't
>> think it should be too tricky to make it work.
> 
> Okay, so far the heavy integration with Plone I saw in places of the 
> codebase worry me. Grok would only need a container type - I'm not sure 
> about the factories and we'd have to do a lot of integration work before 
>  the form bases would be right for us. The workflow support and base 
> classes are not interesting. Given this integration and the state of the 
> tests you mention I'd be tempted to write something new, especially 
> given the amount of dis-integration with Plone that would need to be 
> done before collective.tin would be useful outside a Plone environment.

Apart from the workflow, forms and base classes which are just not 
applicable, I'd reckon the plone/cmf/zope2 specific stuff is limited to 
about 50 lines of python.

container.py - 80% zope3 applicable. Only the base classes are 
redundant. All of the logic is in the adapters anyway.

controller.py - only 1 line uses acquisition, rest is zope3 applicabl

deletion.py - all zope3 (note that the simple delete the row from the 
table version is currently left as an excercise to the reader)

dublincore.py - not applicable

factory.py - all zope3 applicable apart from 1 line, just the localizer, 
I was too lazy to use my own!

form.py - logic should be the same (deferring to factory.py and 
controller.py adapters), but these are UI facing so have become plone 
dependant as I polished.

history.py - versioning - about 3 or 4 aq calls, rest zope3 applicable

item.py - these are just zope2/cmf compatible base classes, you can just 
use object I think.

keytool.py - name/primary key conversion - all zope3

objectmanager.py - not applicable

reindex.py - all zope3

utils.py - the wrapsetup function would need changing. Probably:

def wrapsetup(ob, parent):
     ob.__parent__ = parent
     intermediary = IIntermediary(ob, ob) # fallback to ob
     keytool = IKeyTool(parent)
     ob.__name__ = keytool.keys2name(primary_key_tuple(intermediary))
     return ob

vocabulary.py - all zope3

workflow.py - not applicable. (Though a zope3 compatible DCWorkflow 
would be great!)


Having said that, if you just want one folder with items in a table in 
it then start again, tin is just too complex if you only want to handle 
that use case - the complexity and lines of code come not in the zope2 
compatibility (that is all fairly well contained) but in the support for 
plugable strategies for naming and versioning. It's designed to be 
flexible enough for a SQL first approach where you don't own the 
database you are interfacing with.

Yep, no tests yet. No longer having an intern on the project I'll be 
writing some now ;-) Seriously though, this seems easier now I can get 
sqlite to work (hint: don't use the ancient version that comes with os x 
10.4)

There is certainly a lack of documentation on it. Unfortunately I think 
I'm too bogged down in my project to do much about it at the moment. I'm 
very willing to help anyone else who wants to make use of it though. I'm 
normally on irc during the day.

SQLAlchemy is very well documented though, and extremely powerful. The 
whole sql dialect abstraction layer is wonderful and saves your bacon 
when someone decides to move your project from postgresql to oracle to 
sql server.

Laurence