[Gsoc] Midterm report from Luciano Ramalho on the Wandering Books (Kirbi) project

[Luciano Ramalho]

ABOUT KIRBI

Kirbi is a web application to increase the circulation of books and
DVDs that people have at home, making sharing among friends and
colleagues easier and safer.

Kirbi also aims to demonstrate Grok API usage and best practices in
the context of a non-trivial web application.

WHAT I'VE ACHIEVED SO FAR

I came into this project without any experience coding with or for
Zope 3 or Grok. Part of my contribution has been and will continue to
be give feedback to the Grok developers from the perspective of
someone coming from a Zope 2 background. I am now comfortable coding
with Grok and learning Grok has motivated me to study Zope 3 like
never before. In the course of my studies of Grok I've created two
sample apps (AnimalTree and Adder) and updated a third one (Philipp's
TodoList). They are available at:
- http://svn.zope.org/grok/trunk/doc/examples/
- http://paleosoft.org (demos of AnimalTree and Adder)

The current Kirbi demo which includes:
- views for browsing and searching through a book collection
- demonstrations of the use of different kinds of Indexes (Text for
title/author search, Set for author exact-match search, Field for ISBN
search)
- templates with macros
- handling conditional display of static files (book covers) depending
on their availability
- schema with constraints and invariant (either title or ISBN must be present)
- an XML-RPC interface for importing book data
- validation and handling handling of ISBN-10, ISBN-13
- parsing ISBN to extract probable book language
- generation of filing titles for sorted lists, by moving articles to
the end of the title, in five languages (en, fr, de, es, pt)
- a customized name chooser designed to be fast when importing large
batches of book without ISBN, while generating small and mostly
sequential id numbers
- sample data for 289 books

Besides working on those sample programs, I've organized along with
Dirceu Tiegs the first Brazilian Grok sprint, with our colleagues
Daniel Vainsencher, Ruda Filgueiras, Tomas Correa and Leonardo
Rochael, during which we have revised and extended the Grok tutorial
and created another sample app, Bookshelf. The results of that sprint
have not been published yet but everything we did is in the SVN:

http://svn.zope.org/grok/branches/luciano-tutorial/

Our revised tutorial will be posted on http://paleosoft.org/ for
review later this week.

WHAT I AM CURRENTLY WORKING ON

I am now implementing the external daemon which will fetch book
metadata from Amazon.com. The daemon uses the twisted.internet package
to poll a Kirbi XML-RPC method which returns the list of books in the
Kirbi catalog containing a valid ISBN but no title. Whenever that list
is non-empty, the daemon sets up a queue of ISBNs to fetch from
Amazon.com. As the XML containing the Amazon response is obtained,
it's parsed and the data passed back to Kirbi via XML.

The use of an external daemon is designed to avoid keeping any of the
resource-intensive Zope 3 threads from waiting for remote responses.
The lightweight daemon will do the waiting and push the data into
Kirbi when it is complete and parsed (later I'd like to re-implement
this functionality using REST instead of XML-RPC; if we manage to put
REST support into Grok, this will be a cool demo of it's usage).

After that, the next task is to implement the personal collections,
which will entail integration of principals into the Kirbi model. As I
understand, Grok still provides little help with the Zope 3 principals
API, which will give us an opportunity to contribute functionality to
Grok itself.

WHAT THE PROBLEMS WERE

My main problem has been learning my way around the Zope 3
documentation. Grok's own documentation is sparse at this time, which
requires falling back into Zope 3 documentation, and that is not very
visible and well organized. In the last week or so I've gained
confidence in exploring that documentation, and have proposed to the
Zope 3 community changes to make it more public and interactive,
allowing easier community participation. I also had a hard time
understanding the buildout mechanism, which depends on several other
layers of setup infrastructure, some of which are currently in flux.
Again the relevant documentation is not very visible or well
organized.

Finally, given my inexperience with Grok, my main challenge in coding
Kirbi has been to find the best way of coding things with Grok,
meaning the shortest code that is still very readable. I don't want
Kirbi just to work as app, but to work as a showcase of best practices
as well, so researching, comparing and trying different alternatives
has taken a lot of time. Other Grok examples and doctests are not
always concerned with showing the one preferred way of implementing a
certain feature, but are understandably more concerned with showing
several or even all different ways of doing it. Now that Kirbi has a
functioning and tested buildout, Martijn Faassen (my mentor) has
started providing code reviews, bringing us closer to the goal of
showing the best practices in Grok coding.

Regards,

Luciano Ramalho
student
Library Sciences and Documentation Department
Escola de Comunicações e Artes
Universidade de São Paulo - Brasil