[Grok-dev] Re: Developer scenario

[David Niergarth]

Hi Martijn,

Thanks for your encouraging response.

---- Martijn Faassen <faassen at startifact.com> wrote: 
> That's not entirely fair of course. Recently I've been discovering it's 
> still a lot more common than I thought. Part of the Zope community moved 
> towards "code in the filesystem, data in the ZODB" pattern, but the 
> "code in the ZODB, data in the relational database" is very common 
> still. It is a testament to the low barriers Zope 2 presents to 
> developers. The major drawback is that filesystem tools cannot be 
> employed, and team development using a version control system is much 
> harder. This means applications developed this way can hit a complexity 
> barrier.

I think you're right about there being a good deal of 'Zope 2000'-style development involving relational databases still going on under the radar. (It's hard to emphasize enough how effective we've found it for *rapid* application development.) I'm guessing most of these developers no longer follow the Zope lists because the ZODB development style is so different. Consequently, there are few practical examples for moving from Zope 2 to Zope 3 when using relational database back ends.


> > * For our current nested structure (folders within folders), could
> > the subfolders become submodules or would we need to flatten things
> > out? Our Python Scripts rely on acquisition to locate other scripts
> > and ZSQL Methods (e.g., context.another_script_in_same_folder();
> > context.tools.some_shared_script(), where tools is a folder one or
> > more levels up). Would nested submodules allow most of the calls in
> > Scripts to stay the same?
> 
> I would recommend transforming these to inheritance relationships 
> between views (or models. It depends a little. moving this to MVC will 
> be the trickiest part, though you can get away with doing everything in 
> views for quite a while, I suspect).
> 
> So if you have a folder A with some python scripts, and a folder B 
> inside it, translate it to a view A and a view B that subclasses it. 
> Then associate both views with your application object. Your URLs will 
> change which might not be a problem, but if it is, you could consider 
> making your application object a container, putting a model in it, and 
> then associating view B with that model.

Ah, that gives me something to investigate! Thanks.


> > * Is the psycopg2 database adapter accessible from within grok?
> 
> Grok needs to have a good SQLAlchemy story. SQLAlchemy should be able to 
> do all the SQL you wish. Luckily Zope 3 *does* seem to have a good 
> SQLAlchemy story, so Grok should be able to reuse this information.
> 
> For now, I'd suggest looking into z3c.sqlalchemy:

In another post I wrote about scaling problems when using large databases. But see my comments below also.


> > Hopefully these questions will be relevant for other Zope 2
> > developers who haven't yet moved to Zope 3 (but who "think like
> > caveman, like grok big club"). Thanks for any guidance!
> 
> I think these are very useful questions indeed! I think this scenario is 
> more common than many of us suspect. [...]

Great. This will be very encouraging for the 'Zope 2000' developers who stumble upon this thread while trolling Google for a migration path out of Zope 2.


> We need a number of things:
>
> * a document for Zope 2 developers mapping their existing concepts to 
> how to do it in Grok. How should people change their code?

If we have any luck with this approach, I will summarize what we find.


> * a better low-level relational database integration story. I think 
> we're probably very close to this with z3c.sqlalchemy, it just needs to 
> be researched and documented. We don't even need any of the advanced 
> relational-object mapping stuff that we have been talking about for 
> scenarios like this.

Although the need for ORM support seems obvious (every other framework has one, after all), for 'Zope 2000'-style development, psycopg is the current modus operandi (for PostgreSQL at least) and so switching to an ORM will not be a high priority. Indeed, in our case it's not even practical due to performance/scaling limitations we found with SqlAlchemy. SqlAlchemy support would be great eventually, but it's much less important from our point of view.


> If you are going to start this project, David, it would be *very* useful 
> if you helped us create information like this. Right now you're here 
> first, so unfortunately we can't give you brilliant answers for 
> everything yet. :)

We may try to go this way. If so, I will provide as much information as I can.

Thanks again,

David