[Zope] Discussion: Best practice site struture (LONG)

5 Apr 2002

      Hmm this i quite a long rambling, but I think it sums up most of the 
problems and approaches I have experienced with Zope developing rather 
large sites.

There is often complaints about "best practice" when building Zope 
sites. I try to summ it up here as I see it. Unfortunately it cannot be 
said in a few lines.

Hope somebody will comment on it and is interrested in discussing Zope 
from the birds perspective.

If there is something I have misunderstood, and some best practises I 
have been too dense to catch _please_ tell me.

Btw. English is not my native toungue so there _will_ be speling errors.

--------------------------------

Discussion: Best practice site struture (LONG)
==============================================
But hopefully readable :-)

I have been building sites in Zope for the longest while. Well at least 
a couple of years by now. As my sites have grown larger and more complex 
I have noticed some patterns, and weaknesses in the way Zope is used for 
building sites.

So I have usually found some reasonable solutions to the problems.

Typical structure of site
-------------------------

When you start building sites in Zope you often make a directory 
structure in which you put all your content. Just like you would do in a 
normal webserver like Apache::

    Home/
        acl_users/
            user 1
            user 2
        News/
            article 1
            article 2
        Products/
            Product 1
            Product 2
        About/
            History
            Where to find us
            Adresses
            Jobs
        Contact/

This has the advantage of being conceptually easy. And it works sort of 
ok if there is little relationship between objects in the site. It is 
rather difficult to assign rights to individual users. Ie. how do yo 
assign rights to edit 'articles' but not 'products' etc.

Another more complicated examples is a school. An intuitive structure 
would be::

    Home/
        acl_users/
            teacher 1 # roles = ['Teacher']
            teacher 2 # roles = ['Teacher']
            student 1 # roles = ['Student']
            student 2 # roles = ['Student']
            student 3 # roles = ['Student']
        classes/
            class 1/
                teacher 1/
                    material 1
                    material 2
                student 1/
                    content 1
                    content 2
                student 2/
                    content 3
                    content 4
                subject 1
                subject 2
            class 2/
                teacher 1/
                    material 2
                teacher 2/
                    material 3
                    material 4
                student 2/
                    content 3
                    content 4
                student 3/
                    content 3
                    content 4
                subject 2
                subject 3

But this has several problems.

For one thing it is hell to assing rights with many students, teacher 
and classes.

And a user that is in more than one class will have copies of his 
contents lying around several different places on the site. That breaks 
with the DRY principle. (Don't Repeat Yourself)

So We will need to give the site a new structure. One that is much like 
the one used in the CMF::

    Home/
        acl_users/
            teacher 1 # roles = ['Teacher']
            teacher 2 # roles = ['Teacher']
            student 1 # roles = ['Student']
            student 2 # roles = ['Student']
            student 3 # roles = ['Student']
        members/
            teacher 1/
                material 1
                material 2
            teacher 2/
                material 3
                material 4
            student 1/
                content 1
                content 2
            student 2/
                content 3
                content 4
            student 3/
                content 3
                content 4
        classes/
            class 1/
                subject 1
                subject 2
            class 2/
                subject 2
                subject 3

Here we will need to relate the students with the classes they attend. 
This is typically done by making a list box that saves the id or the 
path of the classes for each user.

This is one of the current problems in Zope. This is a general and often 
reused practice, and should really be refactored out into a common 
module. (I have tried to do this with my mxmRelations product.)

The common approach with a list of id's also has the problem that it is 
one way only. If you store the relations under the student with a list 
class id's, how do you then find which students attends which classes 
from the class' point of view? You have to traverse all students and see 
which students belongs to the class you are interrested in. Either by 
brute force or by using the Catalog.

The CMF structure
-----------------

I have not used the CMF a lot, but as far as i can figure all content is 
stored in a Members folder. This would cause a structure like::

    Home/
        acl_users/
            teacher 1 # roles = ['Teacher']
            teacher 2 # roles = ['Teacher']
            student 1 # roles = ['Student']
            student 2 # roles = ['Student']
            student 3 # roles = ['Student']
        members/
            teacher 1/
                material 1
                material 2
                subject 1
                subject 2
                subject 2
            teacher 2/
                material 3
                material 4
                subject 3
            student 1/
                content 1
                content 2
            student 2/
                content 3
                content 4
            student 3/
                content 3
                content 4
        classes/
            class 1
            class 2

Here the subject that is taught in each class belongs to a single teacher.

This is a particularly bad idea. What if a teacher stops working at the 
school. Copright laws aside, who takes responsibility of his materials 
and subjects? As soon as they are moved to another teachers folder, all 
relations between classes are most likely broken.

Oh and who adds new classes by the way?

Furthermore the subjects under each class sticks out like a sore thumb 
to me. Should subjects really be stored under the classes as the only 
type of objects?

Wouldn't it be more natural to generalize that too?

    Home/
        acl_users/
            teacher 1 # roles = ['Teacher']
            teacher 2 # roles = ['Teacher']
            student 1 # roles = ['Student']
            student 2 # roles = ['Student']
            student 3 # roles = ['Student']
        members/
            teacher 1/
                material 1
                material 2
            teacher 2/
                material 3
                material 4
            student 1/
                content 1
                content 2
            student 2/
                content 3
                content 4
            student 3/
                content 3
                content 4
        subjects/
            subject 1
            subject 2
            subject 2
            subject 3
        classes/
            class 1
            class 2

Model View Controller in Zope
-----------------------------

Most software with a user interface is built using the Model View 
Controller (MVC) principle.

In a standard .php or .asp (.*p) project it is rather easy to seperate 
the three parts as the relational database holds the model, the Views 
are generated on different .*p pages, and the controllers are also built 
on their own .*p page.

In zope the model and the view is part of the same structure if you are 
not carefull.

The taxonomy of a site vs. the model
------------------------------------

In my example here I have restructured the site so that it makes the 
most technical meaning, and avoids redundancy.

But for the users point of view it now has a totally bizarre structure.

Ie. if you are at "home/class 1" and you need to find "content 1" of 
"student 1" how should that be listed? We could have a list of members 
and then the name of all the members in that class. Both teachers and 
students. Perhaps with the type of member beside, like::

    teacher 1 (teacher)
    teacher 2 (teacher)
    student 1 (student)
    student 2 (student)
    student 3 (student)

Then the user would click in the "student 1" link and get to:

    "home/members/user 1/content 1"

*Crunch* <- That is the sound of a mental model crashing. My bet is that 
the user expected to get to::

    "home/class 1/user 1/content 1"

It would perhaps be a little better if the user got to::

    "home/class 1/members/user 1/content 1"

But not a lot.

There is a contradiction between the sites natural taxonomy (Tree 
structure of subjects) and the the way the the site needs to be built to 
avoid redundancy and support relations between objects efficiently.

I believe that the problem with structuring content in Zope is that it 
severely                                                           
advocates mixing Models and views. All the How-To's and the Zope book 
also support this as the "right way" to build Zope sites.

We need a better way to separate models from the views
------------------------------------------------------

When building a site in zope we need two seperate hierachal structures. 
One for the model/content, and another for the view/presentation

Something like::

The site/

    model/
        content 1
        content 2
        content 3

    taxonomy/
        subject 1/
            subject 2/
                subject 3
        subject 4/
            subject 5/
            subject 6/
                subject 7/

The part that should be visible to the general user is the taxonomy. 
This is how the site is viewed and presented. This is where the layout 
and the navigations structure is seen. There should be _no_ content 
objects in a subjects folder. Only methods used for viewing objects, and 
lists of objects.

A school could then have the following structure::

    The site/

        # The model

        acl_users/
            teacher 1 # roles = ['Teacher']
            teacher 2 # roles = ['Teacher']
            student 1 # roles = ['Student']
            student 2 # roles = ['Student']
            student 3 # roles = ['Student']

        subjects/
            subject 1
            subject 2
            subject 2
            subject 3
        classes/
            class 1
            class 2

        members/
            teacher 1/
                material 1
                material 2
            teacher 2/
                material 3
                material 4
            student 1/
                content 1
                content 2
            student 2/
                content 3
                content 4
            student 3/
                content 3
                content 4

        # the taxonomy

        Home/          
            classes/
                class 1/
                    subjects/
                    students/
                        student 1/
                            content 1
                            content 2
                    teachers/
                        teacher 1/
                            material 1
                            material 2
                ... etc.

But then how do we map the content to the structure? There are several 
problems in this.

One way, is to use the catalog to pull out the content from the model 
part of the site. But again this is subject to all the problems 
mentioned above when relating objects.

Another way I have thought about recently is to build in the relations 
in the subject folders. So that that if "subject 1" should present 
"content 1" and       "content 2", there would be generalized methods 
for getting "related" objects like::

    content2.subjectIds()
    content2.subjectValues()
    content2.subjectItems()

Doing this way in Zope right now is really not easy because it lacks a 
general way of relating objects 2 ways. My mxmRelations products takes 
us some of the way, but not quite.

There is also still the problem with the paths to the objects. The 
content should ideally show up as::

    home/subject 1/content 1

and not as::

    home/model/content 1

or::

    home/subject 1/model/content 1

It would be rather trivial to subclass a Folder class and add the 
possibility to and relate objects to it. Also writing the subject*() 
methods would be rather trivial, but if you want to show an object as::

    home/subject 1/content 1

You can unfortunately not do that, as several objects can have the same 
id, all over the model part of the site.

So I guess that the best we can do is to call the content part of the 
site something sensible, and hope that the user doesn't get too confused::

    home/subject 1/content/content 1

This also solves the problem of not having to assign rights in two 
seperate structures.

Short piece of code
-------------------

I would like to show a short piece of code for why this structure can be 
smart.

I have the following taxonomy::

    home/
        class 1
            students
            teachers
        class 2
            students
            teachers

class 1 has student 1, student 2, teacher 1 and teacher 2 related to it. 
So how do I show which teacher a student has? I call::

    I would create a method "teachers()" in the

    student1Teachers = self.class1.subjectValues('teacher')

Shared ownership and the Zmi interface
--------------------------------------

A problem with putting all the shared objects in their own folders is 
that it breaks the sites internal logic for the "Members" of the site. 
Those who edit and update content. They can go to their own folder and 
add objects. But how do they add/edit objects in the shared folders?

If a teacher needs to add or edit a subject that is not in his own 
folder, how does he then get to it?::

        subjects/
            subject 1
            subject 2
            subject 2
            subject 3
        classes/
            class 1
            class 2

        members/
            teacher 1/
                material 1
                material 2
            teacher 2/
                material 3
                material 4
            student 1/
                content 1
                content 2
            student 2/
                content 3
                content 4
            student 3/
                content 3
                content 4

Imagine if the teacher goes to "home/members/teacher 1/manage" to log on 
to the site to be able to edit the content of his folder. All he can see 
is::

    user 1/
        material 1
        material 2

There are no signs of the subjects anywhere in sight. So he must 
manually go to "home/subjects/manage" to add/edit subjects. And for each 
new content type that is available one the site he must do this. That sucks.

The idea of letting a user log on in the root of the management 
interface "home/manage" is also a bad idea. There is probably a lot of 
stuff in it that will confuse most users. Users should really only see 
what they can change.

I believe that the best solution to this problem is to make another tab 
in the zmi in the member folder. It should be directly beside the 
"Contents" tab, and could be called "Shared contents." In it there 
should be a list of the objects that the user has rights to add or edit 
or delete objects in.

So the user from his management interface he might see something like::

    user 1 [Contents]/
        material 1
        material 2
    user 1 [Shared contents]/
        subjects/
            subject 1
            subject 2
            subject 2
            subject 3
        classes/
            class 1
            class 2

That's all folks!

    regards Max M