[Grok-dev] tutorial crashes at 'grokproject Sample'

[Martijn Faassen]

Jean Daniel wrote:
> The same steps, on the stable Ubuntu 9.04, yield a problem, the
> package python2.5-dev is missing: compilation of zope.security can't
> be done.
> 
> I think all newbie will fall for this one while all experienced python
> developers will never notice the problem. I installed the python2.5
> -dev package and it seems better but maybe zope modules other than
> zope.security require the Python.h headers too. Now that the Python.h
> headers are installed on my system, I can't say easily.

It is mentioned in the tutorial (in setting up a virtualenv sidebar and 
in the main text). Note it doesn't happen on Windows as there are binary 
packages available.

> I think this is a bug:
> 
> 1. either in the tutorial which should be modified to make the user
> install the python2.5-dev package (berk),

The tutorial does mention it in several places. I've just expanded the 
text in SVN however in the hope more people will notice it.

> 2. or in grokproject which omit the dependency (can buildout install
> python headers?),

Could theoretically be done by having buildout install a whole Python, 
but we're not going there.

> 3. or in zope.security which seems not to be distributed as a binary,

We deliberately do not distribute binaries for the Linux and Mac OS X 
platforms. These platforms are rather diverse so it's hard to have 
binary eggs that work on all platforms (the internal unicode encoding of 
Python can be 2 byte and 4 byte, for instance) and it's common for 
developers to compile Python packages.

> 4. or in Grok's plan which does not support distribution packages.
> 
> About 4., I know everyone will not agree and it is a difficult one
> indeed. Buildout is good but, at this time, only supports some
> application in one language. Administrators integrating not just Web
> application but streamlining server appliance are outside the scope as
> buildout does not package most of the other tools which can be Varnish
> or Postgresql but also Pacemaker, Net-snmp, Gstreamer, shared
> libraries or kernel modules.

(Actually buildout can be used to install a whole ton of things, 
including C libraries and Varnish with the right buildout recipes. 
Whether this is is a good way to deploy is another question and depends 
on the context.)

There is a difference between the packaging and installation needs of 
(Python) developers and the needs of operating system users. A developer 
needs to be able to independently install the same package for multiple 
projects on the same platform, possibly of different versions. An OS 
distribution tries to install *one* version of a package and has it 
reused for everything installed. There are benefits and drawbacks to 
both approaches.

Note that it is also possible, but underdocumented, to create .deb or 
.rpm packages from a buildout that package all the Python code needed to 
run an application. zc.resourcerelease is generally used to produce such 
buildouts.

Splitting up all of Grok's dependencies into *separate* OS packages is a 
major task with a lot of unanswered questions. We regularly upgrade to 
newer versions of underlying Python libraries in the ZTK. We also need 
to maintain projects which depend on earlier versions of ZTK libraries. 
I'm not sure how packaging solutions from Linux distributions can be 
applied here.

> Nicolas Chauvat does a better job at explaining the problem of
> maintaining servers in production :
> http://www.logilab.org/blogentry/9860

I'm aware of Nicolas's discussion (and have been in personal 
communication about it too). Nicolas has a lot of useful things to say. 
On the other hand, I also think many of the arguments for using OS 
packaging solutions tend to forget about the distinct needs of developers.

I'll also note that buildout is effectively a configuration management 
system focused on software developers (instead of OS distribution 
maintainers).

> There is a python policy for debs, Rick Spencer's Quickly templates
> handle deb package bootstrapping and Launchpad generates deb
> repositories. Since we all have so much free time, we should od just
> that.

If you have a lot of time of course it'd be very much appreciated if you 
investigated this problem more deeply. :)

Regards,

Martijn