-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi everyone, Here's a followup on a docs.zope.org automation task I took over during one of the Zope developer IRC metings[1]. The task was to provide individual package documentation, if it exists, directly underneath docs.zope.org, e.g.: http://docs.zope.org/zope.event/ ... and keep those docs updated from the current trunk documentation. I expanded that task a little bit by generating content that shows the list of packages divided by project and makes it obvious which packages have documentation and which don't. All of that is in a buildout that automatically generates the docs.zope.org website and does away with the single static HTML page with the "dead" new new new new.zope.org design and uses stock Sphinx design. If you visit docs.zope.org now you'll notice that the content on the main page has not changed much, it's just been updated and re-ordered slightly. What's new are those links I prefixed with "Package documentation", those point to the autogenerated index pages for each project. As noted on top of most of these index pages, if a package is listed but not linked it doesn't have documentation the scripts can recognize. Currently the buildout can recognize the standard Sphinx documentation layout with a folder named "doc" or "docs" in the package root and a Sphinx configuration and content therein. The buildout can also use a kludge and pull in content that is documented using "z3c.recipe.sphinxdoc" maintained in a separate buildout, but I'm hoping those project authors would consider using the standard layout as exemplified by e.g. zope.event[2] and championed by e.g. Tres, Christian Theune, and several others (including me). There's a README in the buildout that explains how to do specific tasks[3]. If you have any questions let me know. jens [1] http://docs.zope.org/zopetoolkit/zope-dev/zope-dev-20100727.html [2] http://svn.zope.org/zope.event/trunk/docs/ [3] http://svn.zope.org/docs.zope.org_website/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxUW7gACgkQRAx5nvEhZLLhAgCgsHQeyHi9/xDc9aYvLELlM1/e Z0sAnirP2CO3zQ/MTntWTixDj7MIStgA =gadb -----END PGP SIGNATURE-----
On 07/31/2010 07:22 PM, Jens Vagelpohl wrote:
Here's a followup on a docs.zope.org automation task I took over during one of the Zope developer IRC metings[1]. The task was to provide individual package documentation, if it exists, directly underneath docs.zope.org, e.g.:
Really cool! Wouldn't it be good to put this under /package/zope.event to avoid potential naming conflicts? I realize they're rare, but I can imagine that a project foo could exist that wants to expose its documentation separately from project foo. Perhaps "Zope" would be a good example. :) Regards, Martijn P.S. It'd be good if the ZODB section had at least a link to zodb.org. Links to buildout.org and grok.zope.org might also be useful.
On 8/2/10 13:40 , Martijn Faassen wrote:
On 07/31/2010 07:22 PM, Jens Vagelpohl wrote:
Here's a followup on a docs.zope.org automation task I took over during one of the Zope developer IRC metings[1]. The task was to provide individual package documentation, if it exists, directly underneath docs.zope.org, e.g.:
Really cool!
Wouldn't it be good to put this under /package/zope.event to avoid potential naming conflicts? I realize they're rare, but I can imagine that a project foo could exist that wants to expose its documentation separately from project foo. Perhaps "Zope" would be a good example. :)
Perhaps we should encourage people to upload documentation to packages.python.org ? Wichert.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Martijn,
Really cool!
Wouldn't it be good to put this under /package/zope.event to avoid potential naming conflicts? I realize they're rare, but I can imagine that a project foo could exist that wants to expose its documentation separately from project foo. Perhaps "Zope" would be a good example. :)
The specific desire was to offer URLs http://docs.zope.org/<package>, that's why I created it that way. It's not that hard to add a subdirectory, but I'd rather have consensus first. Me personally I like the straight package URLs without an intermediary subfolder.
P.S. It'd be good if the ZODB section had at least a link to zodb.org. Links to buildout.org and grok.zope.org might also be useful.
Take a look at the "Related Links" on the left hand side, they're all in there except for grok.zope.org, which I just added. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxWsOQACgkQRAx5nvEhZLKZOgCdGGocKi8dFHkAXPP7W1a7Bqn0 0ZcAnRROfJWNE7ET1ngivR2gvnnb8BYC =X2Io -----END PGP SIGNATURE-----
On Saturday, July 31, 2010, Jens Vagelpohl wrote:
Currently the buildout can recognize the standard Sphinx documentation layout with a folder named "doc" or "docs" in the package root and a Sphinx configuration and content therein. The buildout can also use a kludge and pull in content that is documented using "z3c.recipe.sphinxdoc" maintained in a separate buildout, but I'm hoping those project authors would consider using the standard layout as exemplified by e.g. zope.event[2] and championed by e.g. Tres, Christian Theune, and several others (including me).
We have put a lot of energy into putting the full documentation of packages into the package's long description and most packages have this layout (I know, because I did it for most. ;-) So it might be good to use the long description of the package, if the other two methods fail. This would turn on documentation for a lot of packages and we can move them over slowly. But otherwise, it is great to see such great progress on docs.zope.org! Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/2/10 15:51 , Stephan Richter wrote:
We have put a lot of energy into putting the full documentation of packages into the package's long description and most packages have this layout (I know, because I did it for most. ;-) So it might be good to use the long description of the package, if the other two methods fail. This would turn on documentation for a lot of packages and we can move them over slowly.
I'll have to look at that. Currently, the documentation builder does not do any introspection on the package itself, mostly because I do not want to fully install the package and pull in all dependencies. Maybe there's a simple way that does not require full installation. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxWzlMACgkQRAx5nvEhZLJbtgCfQXs8pT87JAVFFzRwoSKnZ4pm QuYAoJ09ArtylS+MUsYXXqmwxo/0Vogr =WBJO -----END PGP SIGNATURE-----
On Monday, August 02, 2010, Jens Vagelpohl wrote:
'll have to look at that. Currently, the documentation builder does not do any introspection on the package itself, mostly because I do not want to fully install the package and pull in all dependencies. Maybe there's a simple way that does not require full installation.
I agree. This does not build the package: python setup.py --long-description Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/2/10 16:36 , Stephan Richter wrote:
On Monday, August 02, 2010, Jens Vagelpohl wrote:
'll have to look at that. Currently, the documentation builder does not do any introspection on the package itself, mostly because I do not want to fully install the package and pull in all dependencies. Maybe there's a simple way that does not require full installation.
I agree. This does not build the package:
python setup.py --long-description
Thanks for the hint, I'll try that. Can you give me a sample package where the long description is supposed to be the main documentation? And what's the output from that? If it's ReST I'd have to find a way to convert it to HTML on the fly... <sigh> jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxW2OoACgkQRAx5nvEhZLLLlwCeKw5Y6x9r3x+WaOb1YMlJtY2j nUIAniBuzDN0MJNKbpXf6y14e+AR/sU7 =8yM8 -----END PGP SIGNATURE-----
On 2 August 2010 22:40, Jens Vagelpohl <jens@dataflake.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 8/2/10 16:36 , Stephan Richter wrote:
On Monday, August 02, 2010, Jens Vagelpohl wrote:
'll have to look at that. Currently, the documentation builder does not do any introspection on the package itself, mostly because I do not want to fully install the package and pull in all dependencies. Maybe there's a simple way that does not require full installation.
I agree. This does not build the package:
python setup.py --long-description
Thanks for the hint, I'll try that. Can you give me a sample package where the long description is supposed to be the main documentation? And what's the output from that? If it's ReST I'd have to find a way to convert it to HTML on the fly... <sigh>
z3c.form, I'd guess. :) For other examples, look at e.g. z3c.caching, plone.caching or plone.testing. You can do: $ python setup.py --long-description | rst2html.py > doc.html to create an HTML file. Martin
On 8/2/10 16:46 , Martin Aspeli wrote:
On 2 August 2010 22:40, Jens Vagelpohl<jens@dataflake.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 8/2/10 16:36 , Stephan Richter wrote:
On Monday, August 02, 2010, Jens Vagelpohl wrote:
'll have to look at that. Currently, the documentation builder does not do any introspection on the package itself, mostly because I do not want to fully install the package and pull in all dependencies. Maybe there's a simple way that does not require full installation.
I agree. This does not build the package:
python setup.py --long-description
Thanks for the hint, I'll try that. Can you give me a sample package where the long description is supposed to be the main documentation? And what's the output from that? If it's ReST I'd have to find a way to convert it to HTML on the fly...<sigh>
z3c.form, I'd guess. :)
Doesn't z3c.form have sphinx documentation? Wichert.
On Monday, August 02, 2010, Jens Vagelpohl wrote:
Thanks for the hint, I'll try that. Can you give me a sample package where the long description is supposed to be the main documentation? And what's the output from that? If it's ReST I'd have to find a way to convert it to HTML on the fly... <sigh>
zope.security (http://pypi.python.org/pypi/zope.security) Once you have docutils installed, you can do: py26 setup.py --long-description | rst2html rst2html have many options that allow you to modify the style, including the ability to specify a stylesheet. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
On Monday, August 02, 2010, Stephan Richter wrote:
py26 setup.py --long-description | rst2html
rst2html have many options that allow you to modify the style, including the ability to specify a stylesheet.
Oh, btw, this is also the method that we used to verify that the long descriptions are proper ReST. ;-) Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/2/10 16:50 , Stephan Richter wrote:
On Monday, August 02, 2010, Stephan Richter wrote:
py26 setup.py --long-description | rst2html
rst2html have many options that allow you to modify the style, including the ability to specify a stylesheet.
Oh, btw, this is also the method that we used to verify that the long descriptions are proper ReST. ;-)
I hereby declare that verifying proper ReST is OUT OF SCOPE for docs.zope.org ;-) jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxW2/QACgkQRAx5nvEhZLJHTQCfW3YchH6YZyJA+JlDgV6J6W7j /1cAn14kcIGJLoIm5iF2KBgiou4t4YCt =0JKd -----END PGP SIGNATURE-----
On 08/02/2010 04:53 PM, Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 8/2/10 16:50 , Stephan Richter wrote:
On Monday, August 02, 2010, Stephan Richter wrote:
py26 setup.py --long-description | rst2html
rst2html have many options that allow you to modify the style, including the ability to specify a stylesheet.
Oh, btw, this is also the method that we used to verify that the long descriptions are proper ReST. ;-)
I hereby declare that verifying proper ReST is OUT OF SCOPE for docs.zope.org ;-)
It's out of scope for PyPI too, what it does is just display the plain text if there are any ReST errors. And when the developer sees that, that's a clue to go and fix the ReST and use the above technique. So as long as there's a clue that "it's not working" we'll be fine. Regards, Martijn
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/2/10 16:48 , Stephan Richter wrote:
On Monday, August 02, 2010, Jens Vagelpohl wrote:
Thanks for the hint, I'll try that. Can you give me a sample package where the long description is supposed to be the main documentation? And what's the output from that? If it's ReST I'd have to find a way to convert it to HTML on the fly... <sigh>
zope.security (http://pypi.python.org/pypi/zope.security)
Once you have docutils installed, you can do:
py26 setup.py --long-description | rst2html
rst2html have many options that allow you to modify the style, including the ability to specify a stylesheet.
OK, I'll see what I can do with that. I'll probably end up using the long description as last fallback. So you will see every package linked to *something*, even if it's just a page with a few words on it. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxW26cACgkQRAx5nvEhZLIczgCfamZ/m01aQt3+h7DnQTDXYpUl /3cAn2Ji0K7MQWHNWZFT/QoBENKZ/r3J =HFeU -----END PGP SIGNATURE-----
On Monday, August 02, 2010, Jens Vagelpohl wrote:
OK, I'll see what I can do with that. I'll probably end up using the long description as last fallback. So you will see every package linked to *something*, even if it's just a page with a few words on it.
Just FYI: If there is no long description (which is very unlikely), then the string "UNKNOWN" is returned from the --long-description call. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/2/10 17:00 , Stephan Richter wrote:
On Monday, August 02, 2010, Jens Vagelpohl wrote:
OK, I'll see what I can do with that. I'll probably end up using the long description as last fallback. So you will see every package linked to *something*, even if it's just a page with a few words on it.
Just FYI: If there is no long description (which is very unlikely), then the string "UNKNOWN" is returned from the --long-description call.
OK, that's an excellent flag to give up and say "this package has no documentation I can deal with". jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxW3gEACgkQRAx5nvEhZLKfJwCfVC1ogbRfnzAXX7UUz8mTDYIq 4W8An25biGvShwZrZXUdFCZdFKVg+GsU =cGph -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/2/10 16:36 , Stephan Richter wrote:
On Monday, August 02, 2010, Jens Vagelpohl wrote:
'll have to look at that. Currently, the documentation builder does not do any introspection on the package itself, mostly because I do not want to fully install the package and pull in all dependencies. Maybe there's a simple way that does not require full installation.
I agree. This does not build the package:
python setup.py --long-description
Can someone tell me how to do that when I am in Python code already? Given the path to the checkout, can I use some setuptools/pkg_resources or pkginfo magic to get at this data? My specific problem is this: Many packages define an intermediate folder, like "src", in which the actual code path is stored. When someone runs "setup.py egg_info" then the .egg-info folder ends up in "src" and not at the root. Trying to use e.g. pkginfo.Develop or pkg_resources.find_distributions with the root folder won't find the .egg-info folder in the "src" subfolder. I'd have to do something horrible like os.walk to find distributions, which I really don't want. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxW41sACgkQRAx5nvEhZLIrzgCeMVQhPaa0QwSZriPxyzTkyApk 6kIAoIbmZ19Rhq4nKOksK5byw5FAwsyg =d3W2 -----END PGP SIGNATURE-----
On 08/02/2010 05:25 PM, Jens Vagelpohl wrote:
python setup.py --long-description
Can someone tell me how to do that when I am in Python code already? Given the path to the checkout, can I use some setuptools/pkg_resources or pkginfo magic to get at this data?
Oh, this is where I always get scared because I always get lost. :) The pkg_resources documentation is here: http://packages.python.org/distribute/pkg_resources.html You can probably create a Distribution object somehow (handwave) from a path (whether that's the path of the package or the path the package is in, not sure). Then maybe 'get_metadata('long_description') on the distribution object, but maybe not as one seems to have to supply metadata when constructing a distribution.. But again, I always get lost here, perhaps you need an Environment or a WorkingSet first, etc. Regards, Martijn
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/2/10 18:26 , Martijn Faassen wrote:
http://packages.python.org/distribute/pkg_resources.html
You can probably create a Distribution object somehow (handwave) from a path (whether that's the path of the package or the path the package is in, not sure).
The issue, just like with pkginfo.Develop, is that I can't find any function or method that finds package information in subfolders. If you look at... http://packages.python.org/distribute/pkg_resources.html#getting-or-creating... ...there's this function find_distributions which supposedly offers subfolder searching, but it just doesn't work. At least it doesn't do what I would expect when I read the documentation. It finds nothing. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxW840ACgkQRAx5nvEhZLK0MgCfWNsSL47dimbU9TlIZNSRq07h HLIAoIYYLdYUsA5PhYrtqhu9EaVk8Nip =GbbC -----END PGP SIGNATURE-----
On 08/02/2010 06:34 PM, Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 8/2/10 18:26 , Martijn Faassen wrote:
http://packages.python.org/distribute/pkg_resources.html
You can probably create a Distribution object somehow (handwave) from a path (whether that's the path of the package or the path the package is in, not sure).
The issue, just like with pkginfo.Develop, is that I can't find any function or method that finds package information in subfolders. If you look at...
http://packages.python.org/distribute/pkg_resources.html#getting-or-creating...
...there's this function find_distributions which supposedly offers subfolder searching, but it just doesn't work. At least it doesn't do what I would expect when I read the documentation. It finds nothing.
Yeah, unfortunately this is generally the point where I find myself being lost too. Regards, Martijn
On Mon, Aug 02, 2010 at 06:34:21PM +0200, Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 8/2/10 18:26 , Martijn Faassen wrote:
http://packages.python.org/distribute/pkg_resources.html
You can probably create a Distribution object somehow (handwave) from a path (whether that's the path of the package or the path the package is in, not sure).
The issue, just like with pkginfo.Develop, is that I can't find any function or method that finds package information in subfolders. If you look at...
http://packages.python.org/distribute/pkg_resources.html#getting-or-creating...
...there's this function find_distributions which supposedly offers subfolder searching, but it just doesn't work. At least it doesn't do what I would expect when I read the documentation. It finds nothing.
Do you have access to an source distribution (tarball or zip) for the package involved? In the source distribution the PKG-INFO file is always at the top level.
jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAkxW840ACgkQRAx5nvEhZLK0MgCfWNsSL47dimbU9TlIZNSRq07h HLIAoIYYLdYUsA5PhYrtqhu9EaVk8Nip =GbbC -----END PGP SIGNATURE----- _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
-- Brian Sutherland
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Brian,
Do you have access to an source distribution (tarball or zip) for the package involved?
In the source distribution the PKG-INFO file is always at the top level.
No, these are trunk checkouts. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxXuq0ACgkQRAx5nvEhZLKuOwCeKf/h1Sb9oxUoTPOBgexPafRC essAoLISBbIoU+fQhzk9w4syHZVtKbxw =trcX -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jens Vagelpohl wrote:
On 8/2/10 18:26 , Martijn Faassen wrote:
You can probably create a Distribution object somehow (handwave) from a path (whether that's the path of the package or the path the package is in, not sure).
The issue, just like with pkginfo.Develop, is that I can't find any function or method that finds package information in subfolders. If you look at...
http://packages.python.org/distribute/pkg_resources.html#getting-or-creating...
...there's this function find_distributions which supposedly offers subfolder searching, but it just doesn't work. At least it doesn't do what I would expect when I read the documentation. It finds nothing.
The 'pkginfo' utility knows how to extract package metadata from stuff installed on the path: http://packages.python.org/pkginfo/distributions.html#introspecting-installe... as well as from checkouts: http://packages.python.org/pkginfo/distributions.html#introspecting-developm... Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkxXRIgACgkQ+gerLs4ltQ4n+QCdFy+tKu+Gjsi0qqNd+fwdEa5K pIkAn1uef9EUaOCib+xKI5UWcJPytKLD =4+bj -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/3/10 00:19 , Tres Seaver wrote:
The 'pkginfo' utility knows how to extract package metadata from stuff installed on the path:
http://packages.python.org/pkginfo/distributions.html#introspecting-installe...
as well as from checkouts:
http://packages.python.org/pkginfo/distributions.html#introspecting-developm...
Just like the counterparts in setuptools this only works if you pass it a path that has a PKG-INFO or .egg-info directly in that folder. The problem is, with many packages those are not in the root folder of the package after doing a setup.py egg_info. Those packages normally use the "package_dir" argument to their setup call in setup.py, but that's information I can't get to, it's in the argument list for the setup call. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxXwEYACgkQRAx5nvEhZLLuYACePk94ugHgdaHXN3kHP2k+91vq jHUAnA9QIvT89QqyILg8vIVQfrnwpoYv =i3gm -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/2/10 15:51 , Stephan Richter wrote:
We have put a lot of energy into putting the full documentation of packages into the package's long description and most packages have this layout (I know, because I did it for most. ;-) So it might be good to use the long description of the package, if the other two methods fail. This would turn on documentation for a lot of packages and we can move them over slowly.
This is now implemented as fallback, take another look at the package listing, like the ZTK list[1]. I had to resort to executing "setup.py - --long-description" in a subprocess, unfortunately. Never found out how to do that programmatically. jens [1] http://docs.zope.org/ztkpackages.html -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxYf0IACgkQRAx5nvEhZLIFegCgg0WwW5KU4pJufwYboaePCcPL 5NAAnjgLa4vG/i2g1m8MPZI2v3g7o9UB =OPU+ -----END PGP SIGNATURE-----
On Tue, Aug 03, 2010 at 10:42:42PM +0200, Jens Vagelpohl wrote:
On 8/2/10 15:51 , Stephan Richter wrote:
We have put a lot of energy into putting the full documentation of packages into the package's long description and most packages have this layout (I know, because I did it for most. ;-) So it might be good to use the long description of the package, if the other two methods fail. This would turn on documentation for a lot of packages and we can move them over slowly.
This is now implemented as fallback, take another look at the package listing, like the ZTK list[1]. I had to resort to executing "setup.py --long-description" in a subprocess, unfortunately. Never found out how to do that programmatically.
From a source checkout? There's no other way. Many packages compute their long_description by running code (which reads and concatenates multiple text files).
jens
Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/3/10 22:59 , Marius Gedminas wrote:
On Tue, Aug 03, 2010 at 10:42:42PM +0200, Jens Vagelpohl wrote:
listing, like the ZTK list[1]. I had to resort to executing "setup.py --long-description" in a subprocess, unfortunately. Never found out how to do that programmatically.
From a source checkout? There's no other way. Many packages compute their long_description by running code (which reads and concatenates multiple text files).
I understand that. But it must be possible to do that programatically in my code. I mean, "setup.py --long-description" obviously executes Python code, which I may be able to execute myself in my current interpreter session by importing and executing stuff from setuptools. Having to invoke another Python interpreter in a subshell because that's too complicated to do any other way is awful. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxYhIEACgkQRAx5nvEhZLLFqwCeNaScimdWCS3sNQv1QgmwuAr8 bR8AnA1+bcXsAdxuvaNP2AhUXD8S+8Qb =ZE+5 -----END PGP SIGNATURE-----
On Tue, Aug 3, 2010 at 11:05 PM, Jens Vagelpohl <jens@dataflake.org> wrote:
I understand that. But it must be possible to do that programatically in my code. I mean, "setup.py --long-description" obviously executes Python code, which I may be able to execute myself in my current interpreter session by importing and executing stuff from setuptools. Having to invoke another Python interpreter in a subshell because that's too complicated to do any other way is awful.
Maybe this gets you a bit further: import os from setuptools.sandbox import run_setup package = os.path.join(os.curdir, 'zope.component') os.chdir(package) run_setup('setup.py', ['--long-description']) os.chdir(os.pardir) The run_setup call seems to dump the output to sys.stdout which might be a bit painful to grab. Hanno
Jens Vagelpohl wrote:
I understand that. But it must be possible to do that programatically in my code. I mean, "setup.py --long-description" obviously executes Python code, which I may be able to execute myself in my current interpreter session by importing and executing stuff from setuptools.
Yeah, this is why setup.py sucks ;-) I've just been doing some build and release tools for a customer that involve this sort of stuff, I resorted to: with nested(Replacer(),OutputCapture()) as (r,o): r.replace('sys.argv',['X','egg_info']) try: sys.path.insert(0,path) curdir = os.getcwd() os.chdir(path) # yuk, but setup.py is yuk anyway! if 'setup' in sys.modules: del sys.modules['setup'] import setup finally: os.chdir(curdir) sys.path.pop(0) ..in my tests, the del from sys.modules is to allow this to by done multiple times in one process. Replacer is http://packages.python.org/testfixtures/mocking.html#the-context-manager OutputCapture looks like: class OutputCapture: def __enter__(self): self.original_stdout = sys.stdout self.original_stderr = sys.stderr self.output = sys.stdout = sys.stderr = StringIO() return self def __exit__(self,*args): sys.stdout = self.original_stdout sys.stderr = self.original_stderr def compare(self,expected): compare(expected.strip(),self.output.getvalue().strip())
Having to invoke another Python interpreter in a subshell because that's too complicated to do any other way is awful.
Yeah, I do resort to this for some things too ;-) http://packages.python.org/execute/use.html cheers, Chris
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Chris,
I've just been doing some build and release tools for a customer that involve this sort of stuff, I resorted to: <snip lots of code>
Yikes. Maybe the subshell isn't so bad after all :-P jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxZKkQACgkQRAx5nvEhZLKQWACgmwW7mgXSQdj4W64aMXXq/PY9 Z2wAoKHgZs6uYAdvDdoR9MzcegrOm6er =EbE3 -----END PGP SIGNATURE-----
Jens Vagelpohl wrote:
I've just been doing some build and release tools for a customer that involve this sort of stuff, I resorted to: <snip lots of code>
Yikes. Maybe the subshell isn't so bad after all :-P
Yeah ;-) Although getting the right python path set up can be a pain... Here's the actual version I use to do an in-process sdist: try: orig_argv = sys.argv orig_stdout,orig_stderr = sys.stdout,sys.stderr curdir = os.getcwd() sys.argv = sys.argv[0:1]+['egg_info', '--tag-svn-revision', '--tag-date', 'sdist'] sys.path.insert(0,location) os.chdir(location) sys.stdout,sys.stderr = StringIO(),StringIO() # yuk, but setup.py is yuk anyway! if 'setup' in sys.modules: del sys.modules['setup'] import setup finally: os.chdir(curdir) sys.path.pop(0) sys.stdout,sys.stderr = orig_stdout,orig_stderr sys.argv = orig_argv cheers, Chris
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/4/10 11:33 , Chris Withers wrote:
Jens Vagelpohl wrote:
I've just been doing some build and release tools for a customer that involve this sort of stuff, I resorted to: <snip lots of code>
Yikes. Maybe the subshell isn't so bad after all :-P
Yeah ;-) Although getting the right python path set up can be a pain...
In my particular case that's not a problem. I simply re-use sys.path from the running interpreter, which is good enough. Something like this: cmd = 'PYTHONPATH="%s" %s setup.py --long-description' % ( sys.executable, ':'.join(sys.path)) status, output = commands.getstatusoutput(cmd) jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxZNUcACgkQRAx5nvEhZLKNtwCgqgZcNkJWZlS+qMkn8Ra9brnZ gcgAnjnA/KaraIL93Ue+CvPlVzxwYs10 =ykyG -----END PGP SIGNATURE-----
On Tuesday, August 03, 2010, Jens Vagelpohl wrote:
This is now implemented as fallback, take another look at the package listing, like the ZTK list[1]. I had to resort to executing "setup.py --long-description" in a subprocess, unfortunately. Never found out how to do that programmatically.
Yeah, this looks good. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
participants (10)
-
Brian Sutherland -
Chris Withers -
Hanno Schlichting -
Jens Vagelpohl -
Marius Gedminas -
Martijn Faassen -
Martin Aspeli -
Stephan Richter -
Tres Seaver -
Wichert Akkerman