[Grok-dev] Google SoC midterm report: Kirbi (Wandering Books) project

[Luciano Ramalho]

ABOUT KIRBI

Kirbi is a web application to increase the circulation of books and
DVDs that people have at home, making media sharing among friends and
colleagues easier and safer.

Kirbi also aims to demonstrate Grok API usage and best practices in
the context of a non-trivial web application.

WHAT I'VE ACHIEVED SO FAR

I came into this project without any experience coding with or for
Zope 3 or Grok. Part of my contribution has been and will continue to
be give feedback to the Grok and Zope 3 developers from the
perspective of someone coming from a Zope 2 background. I am now
comfortable coding with Grok and learning Grok has motivated me to
study Zope 3 like never before. In the course of my studies of Grok
I've created two trivial sample apps (AnimalTree and Adder) and
updated a third one (Philipp's TodoList). They are available at:
- http://svn.zope.org/grok/trunk/doc/examples/
- http://paleosoft.org (demos of AnimalTree and Adder)

My main focus has been developing Kirbi. It's current version includes:

- views for browsing and searching through a book collection

- demonstrations of the use of different kinds of Indexes (Text for
title/author search, Set for exact-match author search, Field for ISBN
search)

- templates with macros

- handling conditional display of static files (book covers) depending
on their availability

- schema with constraints and invariant (either title or ISBN must be present)

- an XML-RPC interface for importing book data

- validation and conversion of ISBN-10, ISBN-13

- parsing ISBN to extract probable book language

- generation of filing titles for sorted lists, by moving articles to
the end of the title, in five languages (en, fr, de, es, pt)

- a customized name chooser designed to be fast when importing large
batches of books without ISBN, while generating small and mostly
sequential id numbers

- sample data for 289 books

Kirbi can be seen here:
http://svn.zope.org/Sandbox/luciano/kirbi/src/kirbi/ (code)
http://paleosoft.org:8080/kirbi/pac/?query=zope (demo)

Besides working on those programs, I've organized along with Dirceu
Tiegs the first Brazilian Grok sprint, with our colleagues Daniel
Vainsencher, Ruda Filgueiras, Tomas Correa and Leonardo Rochael,
during which we have revised and extended the Grok tutorial and
created another sample app, Bookshelf. The results of that sprint have
not been published yet but everything we did is in the SVN:

http://svn.zope.org/grok/branches/luciano-tutorial/

Our revised tutorial will be posted on http://paleosoft.org/ for
review later this week.

WHAT I AM CURRENTLY WORKING ON

I am now implementing the external daemon which will fetch book
metadata from Amazon.com. The daemon uses the twisted.internet package
to poll a Kirbi XML-RPC method which returns the list of books in the
Kirbi catalog containing a valid ISBN but no title. Whenever that list
is non-empty, the daemon sets up a queue of ISBNs to fetch from
Amazon.com. When each XML containing the Amazon response is retrieved
it is parsed and the data passed back to Kirbi via XML-RPC.

The use of an external daemon is designed to avoid having any of the
resource-intensive Zope 3 threads waiting for remote responses. The
lightweight daemon will do the waiting and push the data into Kirbi
when it is complete and parsed (later I'd like to re-implement this
functionality using REST instead of XML-RPC; when we put REST support
into Grok, this will be a cool demo of it's usage).

After that, the next task is to implement the personal collections,
which will entail integration of principals into the Kirbi model. As I
understand, Grok still provides little help with the Zope 3 principals
API, which will give us an opportunity to contribute functionality to
Grok itself.

WHAT THE PROBLEMS WERE

My main problem has been learning my way around the Zope 3
documentation. Grok's own documentation is sparse at this time, which
requires falling back into Zope 3 documentation, which is
comprehensive but not very visible and well organized. In the last
week or so I've gained confidence in exploring that documentation, and
have proposed to the Zope 3 community changes to make it more public
and interactive, allowing easier community participation. I also had a
hard time understanding the buildout mechanism, which depends on
several other layers of setup infrastructure, some of which are
currently in flux. Again the relevant documentation is not very
visible or well organized.

Finally, given my inexperience with Grok, my main challenge in coding
Kirbi has been to find the best way of coding things with Grok,
meaning the shortest code that is still very readable. I don't want
Kirbi just to work as app, but to work as a showcase of best practices
as well, so researching, comparing and trying different alternatives
has taken a lot of time. Other Grok examples and doctests are not
always concerned with showing the one preferred way of implementing a
certain feature, but are understandably more concerned with showing
several or even all different ways of doing it. Now that Kirbi has a
functioning and tested buildout, Martijn Faassen (my mentor) has
started providing code reviews, bringing us closer to the goal of
showing the best practices in Grok coding.

Regards,

Luciano Ramalho
student
Library Sciences and Documentation Department
Escola de Comunicações e Artes
Universidade de São Paulo - Brasil