patterns for using sphinx with the Zope Toolkit?
Hi there, I am interested in creating sphinx-driven documentation for Zope Toolkit packages. I'd like to maintain the documentation for a package (say, zope.component) in that package, in a 'doc' directory. I'm wondering what experiences people have with maintaining Sphinx docs. I've used plain Sphinx before, but are there any buildout recipes people recommend, for instance? It'd also be interesting to explore using Manuel - how would one add manuel-based testing to a Sphinx documentation tree? I'd like to give the priority to testing documentation samples as opposed to doctest-driven testing. I also want to be careful: sometimes the doctest setup fluff tends to distract from clear documentation, and sometimes the effort in composing doctests will slow down writing documentation. I'd therefore want manuel-tested sample code to be incremental. I want to be able to start out with purely untested sample code and then gradually convert *some* samples over to Manuel. How could we support that pattern? Ideas and opinions? This will also help me write the "writing Zope Toolkit documentation" document. :) Regards, Martijn
Martijn Faassen wrote:
Hi there,
I am interested in creating sphinx-driven documentation for Zope Toolkit packages. I'd like to maintain the documentation for a package (say, zope.component) in that package, in a 'doc' directory.
I'm wondering what experiences people have with maintaining Sphinx docs. I've used plain Sphinx before, but are there any buildout recipes people recommend, for instance?
I just use it out of the box after generating some files using "sphinx-quickstart". I suspect just changing the resulting Makefile alias from: SPHINXBUILD = sphinx-build To: SPHINXBUILD = ../../../bin/sphinx-build .. or so, for some buildout usage would work just fine.
It'd also be interesting to explore using Manuel - how would one add manuel-based testing to a Sphinx documentation tree? I'd like to give the priority to testing documentation samples as opposed to doctest-driven testing. I also want to be careful: sometimes the doctest setup fluff tends to distract from clear documentation, and sometimes the effort in composing doctests will slow down writing documentation. I'd therefore want manuel-tested sample code to be incremental. I want to be able to start out with purely untested sample code and then gradually convert *some* samples over to Manuel. How could we support that pattern?
Python samples in Sphinx docs are generated like so: .. code-block:: python a == 1 I did a bit of fooling around with Manuel, because I wanted to make sure that the code blocks in my documentation actually worked, but I wound up in a place where I use Manuel to check only the *syntax* of a subets of the Sphinx code blocks I use. It will do this right out of the box if you read the Manuel docs But I couldn't really figure out a way to do the moral equivalent of this: .. code-block:: python a == 1 .. manuel-expect: True Maybe I missed it. But even so, having Manuel to check even the syntax of code blocks is really useful; I found a couple of errors.
Ideas and opinions? This will also help me write the "writing Zope Toolkit documentation" document. :)
- C
On Sun, Jan 3, 2010 at 02:44, Chris McDonough <chrism@plope.com> wrote:
But I couldn't really figure out a way to do the moral equivalent of this:
.. code-block:: python
a == 1
.. manuel-expect:
True
Maybe I missed it.
I couldn't either. So I added a sphinx module for Manuel, supporting Sphinx testsetup/testcode/testoutput statements. Missing so far is: - support for groups, - in particular the * group for testsetup, - support for having doctest statements mixed with the above, so you can have >>> statements but put output in a testoutput statement, If the code is acceptable to Manuel people, I'll might work on it some more until my usecases work. :) On the other hand I might abandon this path completely, and make all my testing *not* be doctests, but use Sphinx literal-include instead, and run the tests some other way, because as usual doctests are turning out the be a shitload of work for very little benefit. Also if the above code is deemed as being a Good Idea, I will remove all the usage of zope.testing.doctest from Manuel. This is necessary, as one of the things I need from Manuel is a Python 3 port, and I'm not porting zope.testing.doctest to Python 3. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart Regebro wrote: [snip]
Also if the above code is deemed as being a Good Idea, I will remove all the usage of zope.testing.doctest from Manuel. This is necessary, as one of the things I need from Manuel is a Python 3 port, and I'm not porting zope.testing.doctest to Python 3.
I think this is quite independently from the above the consensus anyway, right? Switching over from zope.testing.doctest to plain doctest anywhere seems like a good idea. Regards, Martijn
On Sun, Jan 3, 2010 at 6:53 AM, Martijn Faassen <faassen@startifact.com> wrote:
Lennart Regebro wrote: [snip]
Also if the above code is deemed as being a Good Idea, I will remove all the usage of zope.testing.doctest from Manuel. This is necessary, as one of the things I need from Manuel is a Python 3 port, and I'm not porting zope.testing.doctest to Python 3.
I think this is quite independently from the above the consensus anyway, right? Switching over from zope.testing.doctest to plain doctest anywhere seems like a good idea.
Manuel uses the standard library's doctest almost exclusively. The only place zope.testing.doctest is used is to be compatible with zope.testing's expectations. I just took a stab at removing all references to zope.testing.doctest from Manuel and got it to work using the zope.testing trunk (Manuel needs DocTestFailureException). -- Benji York
On Sun, Jan 3, 2010 at 23:38, Benji York <benji@zope.com> wrote:
I just took a stab at removing all references to zope.testing.doctest from Manuel and got it to work using the zope.testing trunk (Manuel needs DocTestFailureException).
Awesome! That would make it easy to port to Python 3, which I need. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Sat, Jan 2, 2010 at 8:44 PM, Chris McDonough <chrism@plope.com> wrote:
Python samples in Sphinx docs are generated like so:
.. code-block:: python
a == 1
I did a bit of fooling around with Manuel, because I wanted to make sure that the code blocks in my documentation actually worked, but I wound up in a place where I use Manuel to check only the *syntax* of a subets of the Sphinx code blocks I use. It will do this right out of the box if you read the Manuel docs But I couldn't really figure out a way to do the moral equivalent of this:
.. code-block:: python
a == 1
.. manuel-expect:
True
Sphinx and Manuel both support interpreter blocks, so the normal approach will work: >>> a == 1 True Occasionally you want to show some code but hide the assertions about the effects of the code. You can do that by putting the tests in a reST comment after the code. .. code-block:: python a = Foo() b = a.bar() .. make sure the above worked correctly >>> b.baz 42 -- Benji York
On Sun, Jan 3, 2010 at 22:08, Benji York <benji@zope.com> wrote:
Occasionally you want to show some code but hide the assertions about the effects of the code. You can do that by putting the tests in a reST comment after the code.
.. code-block:: python
a = Foo() b = a.bar()
.. make sure the above worked correctly
>>> b.baz 42
That doesn't work for code that will raise an exception. Also, if you want to both run a command and make sure it's output is correct, then you need to do that twice. That means you are in fact not testing the code that is visible, which defeats the whole purpose of doctests. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Sun, Jan 3, 2010 at 4:48 PM, Lennart Regebro <regebro@gmail.com> wrote:
On Sun, Jan 3, 2010 at 22:08, Benji York <benji@zope.com> wrote:
Occasionally you want to show some code but hide the assertions about the effects of the code. You can do that by putting the tests in a reST comment after the code.
.. code-block:: python
a = Foo() b = a.bar()
.. make sure the above worked correctly
>>> b.baz 42
That doesn't work for code that will raise an exception.
Also, if you want to both run a command and make sure it's output is correct, then you need to do that twice.
In both of those cases normal doctest blocks seem appropriate. Calling foo with the wrong parameters raises an exception: >>> foo('wrong') Traceback (most recent call last): ... ValueError: bad parameters Calling the hello function prints a greeting. >>> hello() Hello world! -- Benji York
On Sun, Jan 3, 2010 at 23:14, Benji York <benji@zope.com> wrote:
In both of those cases normal doctest blocks seem appropriate.
Not if you don't want the output in the formatting, or if you don't want the >>> brackets. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart Regebro wrote:
On Sun, Jan 3, 2010 at 23:14, Benji York <benji@zope.com> wrote:
In both of those cases normal doctest blocks seem appropriate.
Not if you don't want the output in the formatting, or if you don't want the >>> brackets.
Yeah. I haven't thought about this much, so it might be bollocks, but I think something like this is what I'm after: .. code-block-setup:: import sys from somepackage.testing import DummyModule sys.modules['models'] = DummyModule() .. code-block:: python :linenos: from models import MyModel from repoze.bfg.view import bfg_view from repoze.bfg.chameleon_zpt import render_template_to_response @bfg_view(name='my_view', request_method='POST', context=MyModel, permission='read') def my_view(request): return {'a':1} .. code-block-teardown:: del sys.modules['models'] Only the code-block would show up. Actually being a code-block would be helpful, too, so we could use the other features of code-blocks, like line numbers. Or something. - C
On Sun, Jan 3, 2010 at 6:43 PM, Chris McDonough <chrism@plope.com> wrote:
Yeah. I haven't thought about this much, so it might be bollocks, but I think something like this is what I'm after:
.. code-block-setup::
import sys from somepackage.testing import DummyModule sys.modules['models'] = DummyModule()
.. code-block:: python :linenos:
from models import MyModel from repoze.bfg.view import bfg_view from repoze.bfg.chameleon_zpt import render_template_to_response
@bfg_view(name='my_view', request_method='POST', context=MyModel, permission='read') def my_view(request): return {'a':1}
.. code-block-teardown:: del sys.modules['models']
Only the code-block would show up. Actually being a code-block would be helpful, too, so we could use the other features of code-blocks, like line numbers. Or something.
If you replace ".. code-block-setup::" and ".. code-block-teardown::" with ".. invisible-code-block:: python" you can do that with Manuel now (http://packages.python.org/manuel/#invisible-code-blocks). -- Benji York
Benji York wrote:
On Sun, Jan 3, 2010 at 6:43 PM, Chris McDonough <chrism@plope.com> wrote:
Yeah. I haven't thought about this much, so it might be bollocks, but I think something like this is what I'm after:
.. code-block-setup::
import sys from somepackage.testing import DummyModule sys.modules['models'] = DummyModule()
.. code-block:: python :linenos:
from models import MyModel from repoze.bfg.view import bfg_view from repoze.bfg.chameleon_zpt import render_template_to_response
@bfg_view(name='my_view', request_method='POST', context=MyModel, permission='read') def my_view(request): return {'a':1}
.. code-block-teardown:: del sys.modules['models']
Only the code-block would show up. Actually being a code-block would be helpful, too, so we could use the other features of code-blocks, like line numbers. Or something.
If you replace ".. code-block-setup::" and ".. code-block-teardown::" with ".. invisible-code-block:: python" you can do that with Manuel now (http://packages.python.org/manuel/#invisible-code-blocks).
Nice. I'll give that a try. - C
On Mon, Jan 4, 2010 at 00:43, Chris McDonough <chrism@plope.com> wrote:
Yeah. I haven't thought about this much, so it might be bollocks, but I think something like this is what I'm after:
.. code-block-setup::
...
.. code-block:: python :linenos:
....
.. code-block-teardown:: del sys.modules['models']
You don't actually have a test in that example. Here is my usecases: Numero Uno1 ======== Here I call a method that will print things to the stdout. I need to make sure that it prints the correct thing, but I do not want that thing to be shown in the document. .. testsetup:: import sys sys.argv = ['setup.py', 'build'] .. testcode:: from setuptools import setup, find_packages setup(name='Porting to Python 3 examples', version=1.0, ) .. testoutput:: :hide: running build ... writing manifest file 'Porting_to_Python_3_examples.egg-info/SOURCES.txt' Numero Due ======== Here I call a method that will raise an error. I need to make sure that it raises the correct thing, but I do not want that thing to be shown in the document. .. testsetup:: import sys sys.argv = ['setup.py', 'build'] .. testcode:: from setuptools import setup, find_packages setup(name='Porting to Python 3 examples', version=1.0, ) .. testoutput:: :hide: Traceback (most recent call last): SystemExit: False Trinity ===== Here I need a from __future__ import, but I want to use the >>> format, because I *do* want to show the result. .. doctest:: >>> from __future__ import division >>> 5/2 2.5 It is of no great importance what the blocks are called, as long as Sphinx recognizes them as code blocks and indents them the same (I don't know Sphinx rules there). I do prefer having a separate "setup" and "teardown" (although I don't use that one currently) before just using invisible code blocks as I find it clearer, even though I realize it's just different names for the same thing, really. I'll not that none of the above works with Manuels latest release. Uno et Due works with trunk because I added Sphinx support yesterday (which should have been added to a branch, but I guess I got a temporary attack of Plone Collective disease), and Trinity I have working because I made a custom extension to Manuel where I override the standard doctest support and use a parser that is hacked to special case __future__ imports. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Martijn Faassen wrote:
Hi there,
I am interested in creating sphinx-driven documentation for Zope Toolkit packages. I'd like to maintain the documentation for a package (say, zope.component) in that package, in a 'doc' directory.
I'm wondering what experiences people have with maintaining Sphinx docs. I've used plain Sphinx before, but are there any buildout recipes people recommend, for instance?
It'd also be interesting to explore using Manuel - how would one add manuel-based testing to a Sphinx documentation tree? I'd like to give the priority to testing documentation samples as opposed to doctest-driven testing. I also want to be careful: sometimes the doctest setup fluff tends to distract from clear documentation, and sometimes the effort in composing doctests will slow down writing documentation. I'd therefore want manuel-tested sample code to be incremental. I want to be able to start out with purely untested sample code and then gradually convert *some* samples over to Manuel. How could we support that pattern?
We've had good success with http://pypi.python.org/pypi/collective.recipe.sphinxbuilder And also http://pypi.python.org/pypi/repoze.sphinx.autointerface for interface docs
Ideas and opinions? This will also help me write the "writing Zope Toolkit documentation" document. :)
Yay :) Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
Martin Aspeli wrote: [snip]
We've had good success with http://pypi.python.org/pypi/collective.recipe.sphinxbuilder
I'm having trouble with this. I'm trying to set up a "narrative_docs" directory in a package but: * the documentation on collective.recipe.sphinxbuilder pypi talks about a bin/sphinxbuilder script. That simply isn't there (the sphinx-build and sphinx-quickstart commands are). Ah, I see it is named after my section. The documentation didn't say that. * I'm supposed to run sphinx-quickstart by hand and "answer a few questions and choose docs-source as your source folder". I'd like to tell about my source folder, but instead it asks me for a "root path for the documentation". I cannot directly control the source path at all, only whether it is the root path or whether it's in root_path/source. * I'd prefer it if I ended up with a sphinx quickstart script that got preconfigured with the choices I made in buildout anyway, or perhaps even run quickstart itself when necessary (but a manual step here is okay and perhaps the sanest) There's also a z3c.recipe.sphinxdoc recipe. This seems to be geared towards producing sphinx documentation from doctests within a Python package. I think to encourage narrative documentation I'd like to separate this documentation from the source tree (though I *would* like to have the *option* to use doc tests). * Do I need to answer 'y' or 'no' to the 'Create Makefile' question and a windows command file? This is definitely frustrating. :) I want to document something about this so that people have more guidance and that our ZTK packages have a similar documentation structure,... Regards, Martijn
On Sun, Jan 3, 2010 at 10:27 PM, Martijn Faassen <faassen@startifact.com> wrote:
This is definitely frustrating. :) I want to document something about this so that people have more guidance and that our ZTK packages have a similar documentation structure,...
So far it seems manuel, the ztk docs and Zope2 all basically use: [docs] recipe = zc.recipe.egg eggs = Sphinx I tried using a specialized recipe at some point, but found it to give no advantage and just add some pain. Hanno
Martijn Faassen wrote:
Martin Aspeli wrote: [snip]
We've had good success with http://pypi.python.org/pypi/collective.recipe.sphinxbuilder
I'm having trouble with this. I'm trying to set up a "narrative_docs" directory in a package but:
* the documentation on collective.recipe.sphinxbuilder pypi talks about a bin/sphinxbuilder script. That simply isn't there (the sphinx-build and sphinx-quickstart commands are). Ah, I see it is named after my section. The documentation didn't say that.
:)
* I'm supposed to run sphinx-quickstart by hand and "answer a few questions and choose docs-source as your source folder". I'd like to tell about my source folder, but instead it asks me for a "root path for the documentation". I cannot directly control the source path at all, only whether it is the root path or whether it's in root_path/source.
I can't really remember what answers we gave, but I know we ran it once and from then on we've been able to do ./bin/sphinx to rebuild from reST sources. The quickstart thing is only necessary when you start a new project, so you should only use it once per package.
* I'd prefer it if I ended up with a sphinx quickstart script that got preconfigured with the choices I made in buildout anyway, or perhaps even run quickstart itself when necessary (but a manual step here is okay and perhaps the sanest) There's also a z3c.recipe.sphinxdoc recipe. This seems to be geared towards producing sphinx documentation from doctests within a Python package. I think to encourage narrative documentation I'd like to separate this documentation from the source tree (though I *would* like to have the *option* to use doc tests).
Yeah, I saw that one, but had the same conclusion.
* Do I need to answer 'y' or 'no' to the 'Create Makefile' question and a windows command file?
This is definitely frustrating. :) I want to document something about this so that people have more guidance and that our ZTK packages have a similar documentation structure,...
This is what we have in our (project) buildout: [sphinx] recipe = collective.recipe.sphinxbuilder source = ${buildout:directory}/docs/source build = ${buildout:directory}/docs/output eggs = ${eggs:main} repoze.sphinx.autointerface ${eggs:main} is a list of eggs that make up our project's main dependencies. We include them so that we can get the automatically generated documentation to work. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
On Sat, Jan 2, 2010 at 2:21 PM, Martijn Faassen <faassen@startifact.com> wrote:
It'd also be interesting to explore using Manuel - how would one add manuel-based testing to a Sphinx documentation tree? I'd like to give the priority to testing documentation samples as opposed to doctest-driven testing.
The Manuel docs use both Sphinx and Manuel, so should be good examples. I don't use a Sphinx recipe at the moment though. I just used zc.recipe.egg:script. See [sphinx-docs-html] and [build-docs] in the Manuel buildout.ini for details (http://svn.zope.org/*checkout*/manuel/trunk/buildout.cfg).
I also want to be careful: sometimes the doctest setup fluff tends to distract from clear documentation, and sometimes the effort in composing doctests will slow down writing documentation.
You can hide setup code using Manuel: http://packages.python.org/manuel/#invisible-code-blocks.
I'd therefore want manuel-tested sample code to be incremental. I want to be able to start out with purely untested sample code and then gradually convert *some* samples over to Manuel. How could we support that pattern?
Manuel lets you specify which pieces of code should be run and which should be ignored (http://packages.python.org/manuel/#ignore). -- Benji York
Martijn Faassen wrote:
I am interested in creating sphinx-driven documentation for Zope Toolkit packages. I'd like to maintain the documentation for a package (say, zope.component) in that package, in a 'doc' directory.
I have a couple of packages that make good use of the my current pattern-that-I'm-aiming-for-with-all-my-packages: http://packages.python.org/testfixtures/ http://packages.python.org/checker/ I aim to support the build and release process in the docs, specifically building the docs and checking the ReST to go to PyPI: http://packages.python.org/checker/development.html#building-the-documentati... I do this with the following buildout, the docs section is largely boilerplate that I copy from project to project changing only the one egg name: https://secure.simplistix.co.uk/svn/Simplistix/checker/trunk/buildout.cfg I use Manuel for testing, wired in firstly to the package's setup.py: https://secure.simplistix.co.uk/svn/Simplistix/checker/trunk/setup.py ...and then I always have a file for testing the docs: https://secure.simplistix.co.uk/svn/Simplistix/testfixtures/trunk/testfixtur... https://secure.simplistix.co.uk/svn/Simplistix/checker/trunk/checker/tests/t... I routinely do development on Windows, Mac and Linux and all of the above work on all of them. hth, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Tue, Jan 12, 2010 at 12:35 PM, Chris Withers <chris@simplistix.co.uk> wrote:
Martijn Faassen wrote:
I am interested in creating sphinx-driven documentation for Zope Toolkit packages. I'd like to maintain the documentation for a package (say, zope.component) in that package, in a 'doc' directory.
I have a couple of packages that make good use of the my current pattern-that-I'm-aiming-for-with-all-my-packages:
I would support adding documentation to packages.python.org It's very easy as Chris show. I am also using a similar pattern for BlueBream. This is how I am uploading documentation for BlueBream: ./bin/sphinxbuilder;./bin/buildout setup . upload_sphinx You can see the Buildout here: http://svn.zope.org/bluebream/trunk/ In "setup.cfg", we can mention the "upload-dir" option like this: [upload_sphinx] upload-dir = docs/output/html BTW, to upload docs, no need to make a new release. Anyone who is having ownership can upload docs. Regards, Baiju M
participants (8)
-
Baiju M -
Benji York -
Chris McDonough -
Chris Withers -
Hanno Schlichting -
Lennart Regebro -
Martijn Faassen -
Martin Aspeli