[Grok-dev] Atom and RSS feeds in Grok

[Brandon Craig Rhodes]

Last week, I helped the author of the Vice syndication system with a
major refactoring that breaks out the Plone-specific parts and leaves
the pure Zope 3 components in a separate package, "vice.outbound.core",
that should be easy to use from Grok.

ME GROK WANTS TO SYNDICATE!

I will now follow up on this work by writing a Grok component that
allows containers and objects to be easily syndicated in multiple feed
formats (currently Atom, RSS, and RSS 2.0), and I need some advice from
all of you fellow Grok people on how to do it.

I will first write this extension on a branch of Grok, so that we can
see what it would look like if it were included in Grok itself.  Because
it seems to me that syndication will be at least as widely used as, say,
the REST classes, I will probably advocate that syndication go directly
into Grok itself.  But if no one else likes that idea, then I can always
copy the work off into a "megrok.syndication" package later once it's
working.

My main question is what syndication should look like.  In one sense, a
feed - say, an Atom feed - is simply another view one can have of a
container, so we might expect something like:

    # example 1a: setting up feeds like we set up Views

    class HerdAtom(grok.AtomFeed):
        grok.context(MammothHerd)
        grok.name('atom')          # so it appears at 'http://app/herd/atom'

        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name

Of course, you might need to supply lots of custom values besides just
"self.title", and, as usual on views, you could do that either through
object attribute setting during update(), or through writing a property.
We would make sure that we provided reasonable defaults for as many
things as we could; looking at Dublin Core metadata when possible, for
example, and certainly calling self.url(self.context) to find out the
URL of the container being syndicated (unless the user overrides us with
a URL of their own that they want returned!).

But while this idea sounds like fun, things get a bit less pretty if we
want to offer several feed formats for a single container.  Imagine that
we wanted that same MammothHerd to be available through all three feed
formats that Vice supports:

    # example 1b: repeating ourselves to make three views

    class HerdAtom(grok.AtomFeed):
        grok.context(MammothHerd)
        grok.name('atom')
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name

    class HerdRSS(grok.RSSFeed):
        grok.context(MammothHerd)
        grok.name('rss')
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name

    class HerdRSS2(grok.RSS2Feed):
        grok.context(MammothHerd)
        grok.name('rss2')
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name

This is obviously silly; we are repeating three times the information
about how to figure out the title of the feed from the properties of the
object.  Maybe the programmer could avoid this by inheriting the feeds
from some fourth class:

    # example 1c: inheriting views from a common parent class

    class HerdFeed(object):
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name

    class HerdAtom(HerdFeed, grok.AtomFeed):
        grok.context(MammothHerd)
        grok.name('atom')

    class HerdRSS(HerdFeed, grok.RSSFeed):
        grok.context(MammothHerd)
        grok.name('rss')

    class HerdRSS2(HerdFeed, grok.RSS2Feed):
        grok.context(MammothHerd)
        grok.name('rss2')

But here we are beginning to wander into the land of multiple
inheritance and four different classes for something that's really quite
simple, which seems to signal that we have gone astray.  Let's think
through this again.

In Vice, one actually thinks of an Atom or RSS feed as a view that takes
any object that can be adapted to IFeed, derives various values from the
IFeed-adapted object (it takes the .title verbatim, for example, but
converts the .last_modified date to the correct string representation
that syndication feeds need), and then passes itself to a template that
knows how that particular feed format is rendered.  By exposing this
detail in Grok, we could have the user turn a container into a feed in
two steps: first they could provide the IFeed adapter with a specially
grokked "Feed" class, and then, separately, provide the views:

    # example 2: one adapter for the data, another for the view itself

    class HerdFeed(grok.Feed):
        grok.context(MammothHerd)
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name

    class HerdAtom(grok.AtomFeed):
        grok.context(MammothHerd)
        grok.name('atom')

    class HerdRSS(grok.RSSFeed):
        grok.context(MammothHerd)
        grok.name('rss')

    class HerdRSS2(grok.RSS2Feed):
        grok.context(MammothHerd)
        grok.name('rss2')

This scheme would at least prevent feed programmers from having to use
multiple inheritance all the time!  But I cannot escape the thought that
this two-storey scheme requires an unreasonable amount of work to make a
container into a feed.  So, as a third and final step, I really think
that the actual feed views should not have to be hand-created, but that
the "grok.Feed" object should have the ability to go and register the
actual views itself:

    # example 3a: have the HerdFeed register the views
    #             by default, /atom, /rss, /rss2 could be created

    class HerdFeed(grok.Feed):
        grok.context(MammothHerd)
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name

By default, if written like the example above, this would make all three
feed types available as "/atom", "/rss", and "/rss2".  The user could
take control by signalling that they wanted some other collection of
feeds available:

    # example 3b: have the HerdFeed register the views that we name

    class HerdFeed(grok.Feed):
        grok.context(MammothHerd)
        grok.feeds(atom='atom', rss2='rss')  # does not offer RSS 1.0
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name

Some would argue that the user should *always* have to supply the
"grok.feeds()" directive (or however we do it; this could also be
supplied in a class attribute, instead of through a new Grok directive)
instead of having "/atom", "/rss", and "/rss2" magically appear as I
suggested in 3a; I am open to arguments both ways.

Even if we agree that 3b is the way to go, and it's certainly the
approach that I want to advocate in this email, how should it work
behind the scenes?  It looks like it could work equally well two
different ways: first, it could do the "two-storey" adapter scheme by
registering itself as the IFeed adapter for its context, and then going
and also registering named Atom and RSS2 adapters atop its context as
well.  When the user went to view "herd/rss2", the view adapter would
wake up, try to adapt its context to IFeed, and thus wake up the
HerdFeed and call it to figure out the feed title and everything else.
But this two-storey scheme really seems excessive to me, even though, as
I have mentioned, it is currently the Vice way of doing things (at the
moment; this whole experience leads me to want to make Vice simpler
too!)  The alternative is for the HerdFeed in 3b to simply register
*itself* as the view for "/atom" and "/rss", and have it dynamically
select which template it uses to render itself by looking (how?) at what
name it was summoned under.

Anyone who has read this far, please provide feedback on which route I
should try taking! :-)

You will note that I have said nothing about how objects themselves
become feed members.  This is because I consider it straightforward; the
developer will simply grab the type he wants to syndicate and give it an
IFeedItem adapter that makes it eligible to appear in a feed.  We could
make them use grok.Adapter for this, but it would be more friendly to
provide a "FeedItem" content type that gets grokked and registered for
them:

    # example of how a feed item gets prepared for the feed

    class MammothItem(grok.FeedItem):
        grok.context(Mammoth)
        def update(self):
            self.title = "The mammoth named %s" % self.context.name

On another subject: I am imagining that the grok.Feed component would
have a property "items" that, by default, iterates across the context
and returns a list of every item that it can successfully adapt to
IFeedItem.  If the user wanted some different traversal scheme, they
could either provide their own "items" property:

    # example 3c: supplying items with a property

    class HerdFeed(grok.Feed):
        grok.context(MammothHerd)
        grok.feeds(atom='atom', rss2='rss')  # does not offer RSS 1.0
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name
        @property
        def items(self):
            return (item for item in self.context if item.is_published)

Or, of course, they could just provide "items" with an instance
attribute computed right in the update():

    # example 3d: supplying items with a simple instance attribute

    class HerdFeed(grok.Feed):
        grok.context(MammothHerd)
        grok.feeds(atom='atom', rss2='rss')  # does not offer RSS 1.0
        def update(self):
            self.title = 'Events in mammoth herd %s' % self.context.name
            self.items = (item for item in self.context if item.is_published)

Finally, it might be common enough for developers to want recursive
iteration that we want to go ahead and provide a RecursiveFeed that
allows this; or maybe the grok.Feed could accept class-attribute options
like "recursive = True" and "depth = 2"?  This is a more advanced
feature that could wait until we have a bit more experience; but, ideas
are welcome in this area too.

Thanks to anyone who has time to respond! :-)

-- 
Brandon Craig Rhodes   brandon at rhodesmill.org   http://rhodesmill.org/brandon