I've just released the first beta of Manuel, my next-generation doctest project. Being that zope-dev has some of the most well-informed users of doctest (and some of the most well-informed non-users of doctest), I'd like to gather feedback from this group before making an announcement to the wider world. Manuel is available from PyPI (http://pypi.python.org/pypi/manuel) with docs, details, and examples at http://packages.python.org/manuel/. I'm interested in any feedback and/or questions you may have be it technical, documentation, or "marketing" (i.e., how do I describe what Manuel does and what benefits it has). -- Benji York Senior Software Engineer Zope Corporation
On Wed, Jun 24, 2009 at 15:00, Benji York <benji@zope.com> wrote:
I've just released the first beta of Manuel, my next-generation doctest project.
Hmm, very interesting, thank you to sharing wth us!
Manuel is available from PyPI (http://pypi.python.org/pypi/manuel) with docs, details, and examples at http://packages.python.org/manuel/.
I'm interested in any feedback and/or questions you may have be it technical, documentation, or "marketing" (i.e., how do I describe what Manuel does and what benefits it has).
1. good idea for manuel.isolation, because it's for us a the main lack of doctests. But why this functionality is in a "documentation" package? 2. we use intensely z3c.testsetup[1], can you explain how use manuel with it? [1] http://pypi.python.org/pypi/z3c.testsetup -- Sebastien Douche <sdouche@gmail.com>
On Wed, Jun 24, 2009 at 12:59 PM, Sebastien Douche<sdouche@gmail.com> wrote:
1. good idea for manuel.isolation, because it's for us a the main lack of doctests. But why this functionality is in a "documentation" package?
Manuel, like doctest, is both for documentation and testing. I believe manuel.isolation is useful in both contexts. ...or I misunderstood what you meant by "But why this functionality is in a 'documentation' package?".
2. we use intensely z3c.testsetup[1], can you explain how use manuel with it?
I've only glanced at z3c.testsetup so I don't know how to use it with Manuel at the moment. I'll look and see if there's a good way to do so. -- Benji York Senior Software Engineer Zope Corporation
Benji York wrote:
I'm interested in any feedback and/or questions you may have be it technical, documentation, or "marketing" (i.e., how do I describe what Manuel does and what benefits it has).
Hi Benji, This looks really great :-) One thing you may have solved already, but if not would make a fantastic addition is a plugin that writes matching sections to a file. So, rather than having the following in a doctest:
f = open(os.path.join('mytempdir','myfile.zcml'),'w') f.write(''' <configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure> '''.strip()) f.close() zope.configuration.xmlconfig.file(join('mytempdir','myfile.zxml'))
I could have: .. file:: myfile.zcml <configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure>
zope.configuration.xmlconfig.file(join('mytempdir','myfile.zxml'))
...with bonus points for telling me how to get this to render nicely with Sphinx into a box with the file contents in it and a caption of the file name :-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Wed, Jun 24, 2009 at 5:06 PM, Chris Withers<chris@simplistix.co.uk> wrote:
One thing you may have solved already, but if not would make a fantastic addition is a plugin that writes matching sections to a file.
So, rather than having the following in a doctest:
f = open(os.path.join('mytempdir','myfile.zcml'),'w') f.write(''' <configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure> '''.strip()) f.close() zope.configuration.xmlconfig.file(join('mytempdir','myfile.zxml'))
I could have:
.. file:: myfile.zcml
<configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure>
There's no super-nice way to do exactly that, but here's a pretty reasonable version: --- begin quote --- Contents of myfile.zcml:: <configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure> .. -> config >>> import tmpfile >>> f = tmpfile.mkstemp() >>> f.write(config) >>> f.close() >>> zope.configuration.xmlconfig.string(config) --- end quote --- Or, a much nicer version that doesn't use a temporary file (note the change from .file to .string) in the code: --- begin quote --- Contents of myfile.zcml:: <configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure> .. -> config >>> zope.configuration.xmlconfig.string(config) --- end quote ---
...with bonus points for telling me how to get this to render nicely with Sphinx into a box with the file contents in it and a caption of the file name :-)
The contents will be in a box, but the file name would be up to you to describe in prose (as shown). I could imagine a plug-in that understood the Sphinx ".. literalinclude:: example.file" directive and made the included lines available in a variable. That might be handy when you *really* want to keep a file on-disk but still include part or all of it in a document. -- Benji York Senior Software Engineer Zope Corporation
Benji York wrote:
--- begin quote ---
Contents of myfile.zcml::
<configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure>
.. -> config
>>> import tmpfile >>> f = tmpfile.mkstemp() >>> f.write(config) >>> f.close()
Okay, but this looks like I'd have to write the above every time I wanted to save a file *and* the saving code would show in the docs, or am I missing something?
I could imagine a plug-in that understood the Sphinx ".. literalinclude:: example.file" directive and made the included lines available in a variable. That might be handy when you *really* want to keep a file on-disk but still include part or all of it in a document.
I kind-of want the opposite of that. I want an easy, clean way to write chunks from my docs to files in a temporary directory such that the code being tested/documented can get at them. This is for testing things that inherently run on files (eg: zc.buildout recipies and the like). In fact, thinking about it, it'd be great if there were two plugins: - one to write files that code being tested could access from disk - one to read files that code being tested had output and check their contents were as expected. Do you see what I mean? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Wed, Jun 24, 2009 at 6:07 PM, Chris Withers<chris@simplistix.co.uk> wrote:
Benji York wrote:
--- begin quote ---
Contents of myfile.zcml::
<configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure>
.. -> config
>>> import tmpfile >>> f = tmpfile.mkstemp() >>> f.write(config) >>> f.close()
Okay, but this looks like I'd have to write the above every time I wanted to save a file
As given, you are correct. A helper function could be written to shorten it to something like this: --- begin quote --- Contents of myfile.zcml:: <configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure> .. -> config >>> file_name = temp_file_helper(config) >>> zope.configuration.xmlconfig.file(file_name) --- end quote ---
*and* the saving code would show in the docs, or am I missing something?
No, the code wouldn't be visible. The four lines of code after ".. -> config" are in a reST comment (".." introduces a comment block).
In fact, thinking about it, it'd be great if there were two plugins:
- one to write files that code being tested could access from disk
I think the version above with the helper does a reasonable job for this use case.
- one to read files that code being tested had output and check their contents were as expected.
I'd do that with something like this: --- begin quote --- Contents of myfile.zcml:: <configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure> .. -> config >>> file, file_name = make_a_named_temp_file() >>> generate_a_file(fn) >>> file.read() == config True --- end quote --- Alternatively, you might make a purpose-built Manuel plug-in to do exactly what you want. Maybe with syntax like... --- begin quote --- Contents of myfile.zcml:: <configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure> .. verify-file: name_of_function_to_do_verification --- end quote --- Where "name_of_function_to_do_verification" would be passed the file name (extracted from the line before the literal block) and the contents of the literal block. Your plug-in could provide a formatter that would show you a nice diff if there is a discrepancy. The code in manuel.capture would be a good starting point for something like that. -- Benji York Senior Software Engineer Zope Corporation
Coming back to this ancient thread... Benji York wrote:
A helper function could be written to shorten it to something like this:
--- begin quote ---
Contents of myfile.zcml::
<configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure>
.. -> config
>>> file_name = temp_file_helper(config) >>> zope.configuration.xmlconfig.file(file_name)
What I'd really like to do is write a Manuel plugin that will be part of testfixtures' [1] TempDirectory and would handle the following: .. topic:: some/dir/myfile.py :class: file :: class MyClass: def something(self): pass ...to write the file.
--- begin quote ---
Contents of myfile.zcml::
<configure xmlns="http://namespaces.zope.org/zope"> blah blah blah </configure>
.. -> config
>>> file, file_name = make_a_named_temp_file() >>> generate_a_file(fn) >>> file.read() == config True
...and: .. topic:: some/dir/myfile.py :class: expected-file :: class MyClass: def something(self): pass ...to check it exists and matches what should be there, emitting meaningful error messages if things are wrong. Aside from looking at manuel.capture, can you give me any pointers as to what I'd need to write to make all the above happen? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote:
What I'd really like to do is write a Manuel plugin that will be part of testfixtures' [1] TempDirectory and would handle the following:
Oops, forgot to include the: [1] http://pypi.python.org/pypi/testfixtures
.. topic:: some/dir/myfile.py :class: file
::
class MyClass: def something(self): pass
...to write the file. <snip> Aside from looking at manuel.capture, can you give me any pointers as to what I'd need to write to make all the above happen?
...in particular, handling the case where the '::' is there as well as the case when it's not. Oh, and I guess I need to worry about dedenting the text before I write it or use it for comparison. Any ideas? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Tue, Sep 8, 2009 at 1:08 PM, Chris Withers <chris@simplistix.co.uk> wrote:
Chris Withers wrote:
What I'd really like to do is write a Manuel plugin that will be part of testfixtures' [1] TempDirectory and would handle the following:
Aside from looking at manuel.capture, can you give me any pointers as to what I'd need to write to make all the above happen?
...in particular, handling the case where the '::' is there as well as the case when it's not.
manuel.capture does both, other than looking at it, I can't think of anything else to add.
Oh, and I guess I need to worry about dedenting the text before I write it or use it for comparison. Any ideas?
I would; manuel.capture does. -- Benji York Senior Software Engineer Zope Corporation
On 08/09/2009 18:04, Chris Withers wrote:
Coming back to this ancient thread...
...and once more, only a year and a half later ;-)
What I'd really like to do is write a Manuel plugin that will be part of testfixtures' TempDirectory and would handle the following:
.. topic:: some/dir/myfile.py :class: file
::
class MyClass: def something(self): pass
...to write the file.
...and:
.. topic:: some/dir/myfile.py :class: expected-file
::
class MyClass: def something(self): pass
...to check it exists and matches what should be there, emitting meaningful error messages if things are wrong.
...this is now a reality: http://packages.python.org/testfixtures/files.html#using-with-manuel cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Fri, Jan 14, 2011 at 7:42 AM, Chris Withers <chris@simplistix.co.uk> wrote:
http://packages.python.org/testfixtures/files.html#using-with-manuel
Very cool. -- Benji York
Benji York wrote:
I've just released the first beta of Manuel, my next-generation doctest project.
Many thanks for the ideas and work you put in manuel!
I'm interested in any feedback and/or questions you may have be it technical, documentation, or "marketing" (i.e., how do I describe what Manuel does and what benefits it has).
I have a technical comment: If you want to use manuel for writing some test logic that works region by region (which I suspect is a rather standard use case), you have to do some boilerplate wiring that involves iterating over region and making sure that you really only evaluate matching regions and format your own results. It would be nice to have a way to create a Manuel object from just some matching expressions and code that applies to a single example. To see what I mean, you might take a look at tl.testing (the cairo module). The code there is split up into testing logic that compares cairo surfaces to images of the expectations (the Test and Result classes) and a Manuel class that just ties it all up. To have something like that Manuel class provided by the manuel package would be quite helpful. That module also defines its own doc file suite, mainly in order to have a test suite factory with all the features of the standard (or rather, zope.testing) doc test suite. To have a more elaborate doc file suite available from manuel would also be good. -- Thomas
participants (5)
-
Benji York -
Benji York -
Chris Withers -
Sebastien Douche -
Thomas Lotze