[Grok-dev] Re: Developer scenario

[Martijn Faassen]

Hey David,

Thanks for presenting this scenario to this list.

David Niergarth wrote:

> Our application is currently in Zope 2.7. All of our content is
> kept in Postgres. We're really only using the ZODB to organize our
> code. The code objects are limited to Page Templates, Python Scripts,
> ZSQL Methods, JavaScript, and Folders. We have one Zope Product,
> which our Python Scripts (in Zope) use to access Python library code
> (outside Zope) that communicates with Postgres. The ZSQL methods in
> Zope are used mostly by Page Templates for populating list boxes,
> etc.; anything heavy (searching, sorting, updating) is run through
> the library code (via the Zope Product). We have no other content in
> the ZODB.
> 
> A typical folder within the Zope 2 ZMI for our app will contain a
> Page Template, several supporting Python Scripts and ZSQL methods,
> and one or more subfolders, which in turn contain Page Templates,
> Python Scripts, and more subfolders.

Not to offend anyone, but I call this structure the 'Zope 2000' 
development pattern. That is, it's using Zope 2 as was common in the 
year 2000 (with some code moving into the filesystem).

That's not entirely fair of course. Recently I've been discovering it's 
still a lot more common than I thought. Part of the Zope community moved 
towards "code in the filesystem, data in the ZODB" pattern, but the 
"code in the ZODB, data in the relational database" is very common 
still. It is a testament to the low barriers Zope 2 presents to 
developers. The major drawback is that filesystem tools cannot be 
employed, and team development using a version control system is much 
harder. This means applications developed this way can hit a complexity 
barrier.

> So given that structure, is grok appropriate? Is it overkill? Is it
> more focused on managing content objects in the ZODB?

Currently Grok only supports managing content objects in the ZODB. We 
are talking about adding relational database support through object 
relational mapping technologies such as sqlalchemy, but we're far from 
there yet. We'll get there if we get someone who wants to drive this 
pattern of development.

Code in the ZODB is completely out of scope for Grok.

> I've started by dumping a site to the file system with FSDump. From
> the grok tutorial, it looks like our Python scripts would become
> methods on the relevant Model or View class. Our library code could
> be accessed directly. But, ...

Right, that would be the direction I would take. A Zope 2 folder 
translates to a model or view class surprisingly well.

> * For our current nested structure (folders within folders), could
> the subfolders become submodules or would we need to flatten things
> out? Our Python Scripts rely on acquisition to locate other scripts
> and ZSQL Methods (e.g., context.another_script_in_same_folder();
> context.tools.some_shared_script(), where tools is a folder one or
> more levels up). Would nested submodules allow most of the calls in
> Scripts to stay the same?

I would recommend transforming these to inheritance relationships 
between views (or models. It depends a little. moving this to MVC will 
be the trickiest part, though you can get away with doing everything in 
views for quite a while, I suspect).

So if you have a folder A with some python scripts, and a folder B 
inside it, translate it to a view A and a view B that subclasses it. 
Then associate both views with your application object. Your URLs will 
change which might not be a problem, but if it is, you could consider 
making your application object a container, putting a model in it, and 
then associating view B with that model.

> * Is there a way to handle ZSQL? (I've not been able to find any
> examples.)

> * Is the psycopg2 database adapter accessible from within grok?

Grok needs to have a good SQLAlchemy story. SQLAlchemy should be able to 
do all the SQL you wish. Luckily Zope 3 *does* seem to have a good 
SQLAlchemy story, so Grok should be able to reuse this information.

For now, I'd suggest looking into z3c.sqlalchemy:

http://svn.zope.org/z3c.sqlalchemy/trunk/

> Hopefully these questions will be relevant for other Zope 2
> developers who haven't yet moved to Zope 3 (but who "think like
> caveman, like grok big club"). Thanks for any guidance!

I think these are very useful questions indeed! I think this scenario is 
more common than many of us suspect. We need a number of things:

* a document for Zope 2 developers mapping their existing concepts to 
how to do it in Grok. How should people change their code?

* a better low-level relational database integration story. I think 
we're probably very close to this with z3c.sqlalchemy, it just needs to 
be researched and documented. We don't even need any of the advanced 
relational-object mapping stuff that we have been talking about for 
scenarios like this.

If you are going to start this project, David, it would be *very* useful 
  if you helped us create information like this. Right now you're here 
first, so unfortunately we can't give you brilliant answers for 
everything yet. :)

Regards,

Martijn