[Zope3-checkins] SVN: Zope3/trunk/ First stab of making the static
version of the apidoc actually work. It
Stephan Richter
srichter at cosmos.phy.tufts.edu
Wed Oct 26 12:51:16 EDT 2005
Log message for revision 39640:
First stab of making the static version of the apidoc actually work. It
turned out that wget was not good enough for our case and did not rewrite
all links that need to be rewritten.
So I developed a custom retrieval script. The advantage is that we can use
the publisher directly, instead of brining up a Web server. Otherwise the
script is a bit of a mess; no options, no documentation and lots of
spaghetti code. I check it in in hope that people will help me improve it.
While I have not made a complete run yet, about 10500 links are correctly
handled, which is about the first and second tier of links. This easily
includes all links directly available from the menus.
Trying to generate the static version of the apidoc has surfaced several
bugs, especially related to utility names. I will fix those during the
freeze stage.
Changed:
U Zope3/trunk/src/zope/app/apidoc/browser/configure.zcml
U Zope3/trunk/src/zope/app/apidoc/browser/details_macros.pt
U Zope3/trunk/src/zope/app/apidoc/browser/menu_macros.pt
A Zope3/trunk/src/zope/app/apidoc/browser/static_contents.pt
U Zope3/trunk/src/zope/app/apidoc/browser/static_menu_macros.pt
U Zope3/trunk/src/zope/app/apidoc/browser/utilities.js
U Zope3/trunk/src/zope/app/apidoc/codemodule/browser/README.txt
U Zope3/trunk/src/zope/app/apidoc/codemodule/browser/menu.py
U Zope3/trunk/src/zope/app/apidoc/codemodule/browser/static_menu.pt
U Zope3/trunk/src/zope/app/apidoc/ifacemodule/configure.zcml
U Zope3/trunk/src/zope/app/apidoc/ifacemodule/presentation_macros.pt
A Zope3/trunk/src/zope/app/apidoc/static.py
A Zope3/trunk/utilities/static-apidoc
-=-
Modified: Zope3/trunk/src/zope/app/apidoc/browser/configure.zcml
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/browser/configure.zcml 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/browser/configure.zcml 2005-10-26 16:51:16 UTC (rev 39640)
@@ -9,43 +9,43 @@
name="apidoc_macros"
permission="zope.View"
class=".macros.APIDocumentationMacros"
- allowed_interface="zope.interface.common.mapping.IItemMapping"
+ allowed_interface="zope.interface.common.mapping.IItemMapping"
/>
-
- <page
+
+ <page
for="*"
name="menu_macros"
permission="zope.View"
template="menu_macros.pt"
/>
- <page
+ <page
for="*"
name="static_menu_macros"
permission="zope.View"
template="static_menu_macros.pt"
/>
- <page
+ <page
for="*"
name="details_macros"
permission="zope.View"
template="details_macros.pt"
/>
- <resource
- name="utilities.js"
- file="utilities.js"
+ <resource
+ name="utilities.js"
+ file="utilities.js"
/>
- <resource
- name="harrow.png"
- file="harrow.png"
+ <resource
+ name="harrow.png"
+ file="harrow.png"
/>
- <resource
- name="varrow.png"
- file="varrow.png"
+ <resource
+ name="varrow.png"
+ file="varrow.png"
/>
<pages
@@ -68,7 +68,7 @@
<page
name="contents.html"
template="contents.pt" />
-
+
</pages>
<!-- Static apidoc -->
@@ -78,7 +78,6 @@
class=".apidoc.APIDocumentationView"
permission="zope.app.apidoc.UseAPIDoc">
-
<page
name="static.html"
template="static_index.pt" />
@@ -90,10 +89,10 @@
<page
name="staticmenu.html"
template="static_menu.pt" />
-
+
<page
name="staticcontents.html"
- template="contents.pt" />
+ template="static_contents.pt" />
</pages>
@@ -105,7 +104,7 @@
for="zope.app.preference.interfaces.IPreferenceGroup"
class=".preference.APIDocPreferencesTree"
permission="zope.View"
- attribute="apidocTree"
+ attribute="apidocTree"
/>
<page
Modified: Zope3/trunk/src/zope/app/apidoc/browser/details_macros.pt
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/browser/details_macros.pt 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/browser/details_macros.pt 2005-10-26 16:51:16 UTC (rev 39640)
@@ -3,19 +3,14 @@
<html
xmlns="http://www.w3.org/1999/xhtml"
xml:lang="en"
- lang="en"
+ lang="en"
i18n:domain="zope">
<head>
- <title metal:define-slot="title" i18n:translate="">Z3 UI</title>
+ <title metal:define-slot="title" i18n:translate="">Zope 3 apidoc</title>
- <style type="text/css"
- media="all"
- tal:content=
- "string: @import url(${context/++resource++apidoc.css});"
- >
- @import url(/++resource++apidoc.css);
- </style>
+ <link type="text/css" rel="stylesheet" media="all" href=""
+ tal:attributes="href context/++resource++apidoc.css" />
<script type="text/javascript" src="utilities.js"
tal:attributes="src string:${context/++resource++utilities.js}" >
@@ -47,7 +42,7 @@
style="vertical-align: middle"
tal:condition="not:show"
tal:attributes="id string:${elementId}.arrow" />
-
+
<img src="/@@/varrow.png" border="0" id=".arrow"
style="vertical-align: middle"
tal:condition="show"
Modified: Zope3/trunk/src/zope/app/apidoc/browser/menu_macros.pt
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/browser/menu_macros.pt 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/browser/menu_macros.pt 2005-10-26 16:51:16 UTC (rev 39640)
@@ -3,22 +3,15 @@
<html
xmlns="http://www.w3.org/1999/xhtml"
xml:lang="en"
- lang="en"
+ lang="en"
i18n:domain="zope">
<head>
- <!-- Waaa -->
- <title metal:define-slot="title" i18n:translate="">Z3 UI</title>
+ <title metal:define-slot="title" i18n:translate="">Zope 3 apidoc</title>
+ <link type="text/css" rel="stylesheet" media="all" href=""
+ tal:attributes="href context/++resource++apidoc.css" />
- <style type="text/css"
- media="all"
- tal:content=
- "string: @import url(${context/++resource++apidoc.css});"
- >
- @import url(/++resource++apidoc.css);
- </style>
-
<metal:block define-slot="headers" />
<metal:block define-slot="style_slot" />
<metal:block define-slot="ecmascript_slot" />
Added: Zope3/trunk/src/zope/app/apidoc/browser/static_contents.pt
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/browser/static_contents.pt 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/browser/static_contents.pt 2005-10-26 16:51:16 UTC (rev 39640)
@@ -0,0 +1,36 @@
+<html metal:use-macro="views/apidoc_macros/details"
+ i18n:domain="zope">
+<body metal:fill-slot="contents">
+
+ <h1 i18n:translate="">Zope 3 API Documentation</h1>
+
+ <p i18n:translate="">Welcome to the Zope 3 API documentation tool.
+ The documentation provided here is separated in several discrete
+ documentation modules. You can see the list of available modules
+ in the top-left box on your screen. When you click on a module,
+ the module's menu will appear below. In the menu you have then
+ navigational means to access the documentation content of the
+ module.</p>
+
+ <p i18n:translate="">Modules are usually depending on each other by
+ using links that create references across all modules. Each module
+ has a "theme" that it follows, which was designed to aid the developer
+ directly to the desired information. Below you see a short description
+ of every module.</p>
+
+ <tal:omit-tag repeat="module view/getModuleList">
+
+ <h2>
+ <a target="menu" href=""
+ tal:attributes="href string:./${module/name}/@@staticmenu.html"
+ tal:content="module/title" i18n:translate="">Module Title</a>
+ </h2>
+
+ <p tal:replace="structure module/description" i18n:translate="">
+ Module Description
+ </p>
+
+ </tal:omit-tag>
+
+</body>
+</html>
Property changes on: Zope3/trunk/src/zope/app/apidoc/browser/static_contents.pt
___________________________________________________________________
Name: svn:eol-style
+ native
Modified: Zope3/trunk/src/zope/app/apidoc/browser/static_menu_macros.pt
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/browser/static_menu_macros.pt 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/browser/static_menu_macros.pt 2005-10-26 16:51:16 UTC (rev 39640)
@@ -7,17 +7,11 @@
i18n:domain="zope">
<head>
- <!-- Waaa -->
- <title metal:define-slot="title" i18n:translate="">Z3 UI</title>
+ <title metal:define-slot="title" i18n:translate="">Zope 3 apidoc</title>
+ <link type="text/css" rel="stylesheet" media="all" href=""
+ tal:attributes="href string:${context/++resource++apidoc.css}" />
- <style type="text/css"
- media="all"
- tal:content=
- "string: @import url(${context/++resource++apidoc.css});"
- >
- @import url(/++resource++apidoc.css);
- </style>
<script type="text/javascript" src="utilities.js"
tal:attributes="src string:${context/++resource++utilities.js}" >
</script>
Modified: Zope3/trunk/src/zope/app/apidoc/browser/utilities.js
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/browser/utilities.js 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/browser/utilities.js 2005-10-26 16:51:16 UTC (rev 39640)
@@ -21,7 +21,7 @@
}
// Reference the style ...
- if (element.style) {
+ if (element.style) {
style = element.style;
}
@@ -36,11 +36,11 @@
// Change the display style
if (style.display == 'none') {
style.display = '';
- switchImage(id, 'varrow.png');
+ switchImage(id, 'varrow.png');
}
else {
style.display = 'none'
- switchImage(id, 'harrow.png');
+ switchImage(id, 'harrow.png');
}
}
@@ -66,7 +66,7 @@
{
if (searchtext.length == 0) found_sets = new Array();
var searchindex = searchtext.length - 1;
-
+
// process backspace i.e search string gets shorter
if (found_sets.length > searchindex)
{ rubbish = found_sets.pop();
@@ -78,7 +78,7 @@
}
return;
}
-
+
var reslist = document.getElementById('resultlist');
var children = reslist.getElementsByTagName('div') //reslist.childNodes;
var element;
@@ -96,7 +96,7 @@
refelement = subelement[0]; // get one a element
comparetext = refelement.firstChild.nodeValue; // get textnode of a element
compareresult = comparetext.search(searchtext);
-
+
if (compareresult != -1)
{element.style.display='block';
resultarray[itemcount] = element.getAttribute("id");
@@ -108,9 +108,9 @@
function simplegetSearchResult(searchtext)
{
-
+
var searchindex = searchtext.length - 1;
-
+
var reslist = document.getElementById('resultlist');
var children = reslist.getElementsByTagName('div') //reslist.childNodes;
var element;
@@ -128,7 +128,7 @@
compareresult = comparetext.search(searchtext);
if (compareresult != -1)
{element.style.display='block';}
-
+
}
}
@@ -149,13 +149,12 @@
for (var n = 0; n < children.length; n++) {
var element = children[n];
if (found==1) {
- if ( treeiddepth < element.getAttribute("treedepth") ) {
- element.style.display = action;
+ if ( treeiddepth < element.getAttribute("treedepth") ) {
+ element.style.display = action;
var elid = element.getAttribute("id");
- if (document.getElementById("i"+elid) != null) {
- var subimg = document.getElementById("i"+elid);
-
-
+ if (document.getElementById("i"+elid) != null) {
+ var subimg = document.getElementById("i"+elid)
+
if (action=="none" && subimg.src.search('minus') != -1) {
subimg.src = subimg.src.replace('minus', 'plus');
}
@@ -181,4 +180,3 @@
}
}
}
-
Modified: Zope3/trunk/src/zope/app/apidoc/codemodule/browser/README.txt
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/codemodule/browser/README.txt 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/codemodule/browser/README.txt 2005-10-26 16:51:16 UTC (rev 39640)
@@ -91,7 +91,7 @@
Now that we have the details class we can just access the various methods:
`getBases()`
-~~~~~~~~~~~
+~~~~~~~~~~~~
Get all bases of this class.
@@ -301,7 +301,7 @@
containment root.
`ifaceURL(value, field, rootURL)`
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This method converts the string value of the field to an interface and then
crafts a documentation URL for it:
@@ -382,7 +382,7 @@
>>> from zope.app.apidoc.codemodule.browser import introspector
`getTypeLink(type)`
-~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~
This little helper function returns the path to the type class:
Modified: Zope3/trunk/src/zope/app/apidoc/codemodule/browser/menu.py
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/codemodule/browser/menu.py 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/codemodule/browser/menu.py 2005-10-26 16:51:16 UTC (rev 39640)
@@ -34,8 +34,7 @@
>>> from zope.app import zapi
>>> from zope.app.apidoc.codemodule.class_ import Class
>>> from zope.app.apidoc.interfaces import IDocumentationModule
-
-
+
>>> cm = zapi.getUtility(IDocumentationModule, 'Code')
>>> mod = cm['zope']['app']['apidoc']['codemodule']['browser']
@@ -59,28 +58,28 @@
>>> menu.context = None
Testing the method with various inputs.
-
+
>>> menu.request = TestRequest(form={'path': 'Foo'})
>>> info = menu.findClasses()
>>> pprint(info)
[{'path': 'zope.app.apidoc.codemodule.browser.Foo',
- 'url': 'http://127.0.0.1/zope/app/apidoc/codemodule/browser/Foo'},
+ 'url': 'http://127.0.0.1/zope/app/apidoc/codemodule/browser/Foo/'},
{'path': 'zope.app.apidoc.codemodule.browser.Foo2',
- 'url': 'http://127.0.0.1/zope/app/apidoc/codemodule/browser/Foo2'}]
-
+ 'url': 'http://127.0.0.1/zope/app/apidoc/codemodule/browser/Foo2/'}]
+
>>> menu.request = TestRequest(form={'path': 'o2'})
>>> info = menu.findClasses()
>>> pprint(info)
[{'path': 'zope.app.apidoc.codemodule.browser.Foo2',
- 'url': 'http://127.0.0.1/zope/app/apidoc/codemodule/browser/Foo2'}]
+ 'url': 'http://127.0.0.1/zope/app/apidoc/codemodule/browser/Foo2/'}]
-
+
>>> menu.request = TestRequest(form={'path': 'Blah'})
>>> info = menu.findClasses()
>>> pprint(info)
[{'path': 'zope.app.apidoc.codemodule.browser.Blah',
- 'url': 'http://127.0.0.1/zope/app/apidoc/codemodule/browser/Blah'}]
+ 'url': 'http://127.0.0.1/zope/app/apidoc/codemodule/browser/Blah/'}]
"""
path = self.request.get('path', None)
@@ -93,21 +92,21 @@
klass = zapi.traverse(classModule, p.replace('.', '/'))
results.append(
{'path': p,
- 'url': zapi.absoluteURL(klass, self.request)
+ 'url': zapi.absoluteURL(klass, self.request) + '/'
})
results.sort(lambda x, y: cmp(x['path'], y['path']))
return results
-
+
def findAllClasses(self):
-
- """Find all classes
+ """Find all classes
+
Examples::
>>> from zope.app import zapi
>>> from zope.app.apidoc.codemodule.class_ import Class
>>> from zope.app.apidoc.interfaces import IDocumentationModule
-
-
+
+
>>> cm = zapi.getUtility(IDocumentationModule, 'Code')
>>> mod = cm['zope']['app']['apidoc']['codemodule']['browser']
@@ -131,7 +130,7 @@
>>> menu.context = None
Testing the method with various inputs.
-
+
>>> menu.request = TestRequest(form={'path': 'Foo'})
>>> info = menu.findAllClasses()
@@ -150,6 +149,6 @@
'counter': counter
})
counter += 1
-
+
results.sort(lambda x, y: cmp(x['path'], y['path']))
return results
Modified: Zope3/trunk/src/zope/app/apidoc/codemodule/browser/static_menu.pt
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/codemodule/browser/static_menu.pt 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/codemodule/browser/static_menu.pt 2005-10-26 16:51:16 UTC (rev 39640)
@@ -10,16 +10,17 @@
</div>
<form action="#" method="post" name="searchform" >
<br />
- <input type="text" name="path"
+ <input type="text" name="path"
style="font-size: 80%; width=95%" />
-
- <br />
- <input type="button" name="Find" value="Find"
- onClick="javascript:simplegetSearchResult(document.searchform.path.value)"
+
+ <br />
+ <input type="button" name="Find" value="Find"
+ onClick="
+ javascript:simplegetSearchResult(document.searchform.path.value)"
i18n:attributes="value find-button" style="font-size: 80%"/>
-
+
<!--
- <input type="submit" name="SUBMIT" value="Find"
+ <input type="submit" name="SUBMIT" value="Find"
i18n:attributes="value find-button" style="font-size: 80%"/>
-->
</form>
@@ -31,12 +32,13 @@
<div>
<span i18n:translate="">Search results:</span><br /><br />
- </div>
-
+ </div>
+
<div tal:define="pyclasses view/findAllClasses"
tal:condition="pyclasses" id="resultlist">
- <div tal:repeat="info pyclasses" style="display:block" tal:attributes="id info/counter">
+ <div tal:repeat="info pyclasses" style="display:block"
+ tal:attributes="id info/counter">
<a href="" target="main"
tal:attributes="href info/url"
tal:content="info/path">
Modified: Zope3/trunk/src/zope/app/apidoc/ifacemodule/configure.zcml
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/ifacemodule/configure.zcml 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/ifacemodule/configure.zcml 2005-10-26 16:51:16 UTC (rev 39640)
@@ -21,9 +21,9 @@
name="interface_macros"
permission="zope.View"
class=".macros.InterfaceDetailsMacros"
- allowed_interface="zope.interface.common.mapping.IItemMapping"
+ allowed_interface="zope.interface.common.mapping.IItemMapping"
/>
-
+
<browser:page
for="*"
name="iface_macros"
@@ -77,7 +77,7 @@
<preferenceGroup
id="apidoc.InterfaceDetails"
schema=".interfaces.IInterfaceDetailsPreferences"
- title="Interface Details"
+ title="Interface Details"
/>
</configure>
Modified: Zope3/trunk/src/zope/app/apidoc/ifacemodule/presentation_macros.pt
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/ifacemodule/presentation_macros.pt 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/ifacemodule/presentation_macros.pt 2005-10-26 16:51:16 UTC (rev 39640)
@@ -50,7 +50,7 @@
tal:condition="iface">
<i i18n:translate="">layer:</i>
<a href=""
- tal:attributes="href
+ tal:attributes="href
string:../${iface/module}.${iface/name}/apiindex.html">
<metal:block use-macro="context/@@interface_macros/ifacename" />
</a>
@@ -60,8 +60,8 @@
<i i18n:translate="">factory path:</i>
<a href=""
tal:condition="View/factory/referencable"
- tal:attributes="href
- string: ../../Code/${View/factory/url}/index.html"
+ tal:attributes="href
+ string:../../Code/${View/factory/url}/index.html"
tal:content="View/factory/path" />
<span
tal:condition="not:View/factory/referencable"
@@ -76,7 +76,7 @@
<div tal:condition="View/factory/resource">
<i i18n:translate="">resource:</i>
<a href=""
- tal:attributes="href
+ tal:attributes="href
string: /@@/${View/factory/resource}"
tal:content="View/factory/resource" />
</div>
Added: Zope3/trunk/src/zope/app/apidoc/static.py
===================================================================
--- Zope3/trunk/src/zope/app/apidoc/static.py 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/src/zope/app/apidoc/static.py 2005-10-26 16:51:16 UTC (rev 39640)
@@ -0,0 +1,258 @@
+##############################################################################
+#
+# Copyright (c) 2005 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Retrieve Static APIDOC
+
+$Id$
+"""
+__docformat__ = "reStructuredText"
+
+import os
+import sys
+import time
+import urllib2
+import HTMLParser
+
+import zope.testbrowser
+import mechanize
+
+from zope.app.testing import functional
+
+# Setup the user feedback detail level.
+VERBOSITY = 4
+
+VERBOSITY_MAP = {1: 'ERROR', 2: 'WARNING', 3: 'INFO'}
+
+USE_PUBLISHER = True
+
+URL = 'http://localhost:8080/'
+
+START_PAGE = '++apidoc++/static.html'
+
+BASE_DIR = 'apidoc'
+
+# A mapping of HTML elements that can contain links to the attribute that
+# actually contains the link
+urltags = {
+ "a": "href",
+ "area": "href",
+ "frame": "src",
+ "iframe": "src",
+ "link": "href",
+ "img": "src",
+ "script": "src",
+}
+
+# Additional URLs that are not referenced in the HTML, but are still used, via
+# Javascript, for example.
+additionalURLs = [
+ '@@/varrow.png',
+ '@@/harrow.png',
+ ]
+
+def getMaxWidth():
+ try:
+ import curses
+ except ImportError:
+ pass
+ else:
+ try:
+ curses.setupterm()
+ cols = curses.tigetnum('cols')
+ if cols > 0:
+ return cols
+ except curses.error:
+ pass
+ return 80
+
+def cleanURL(url):
+ """Clean a URL from parameters."""
+ if '?' in url:
+ url = url.split('?')[0]
+ if '#' in url:
+ url = url.split('#')[0]
+ return url
+
+def completeURL(url):
+ """Add file to URL, if not provided."""
+ if url.endswith('/'):
+ url += '@@index.html'
+ if '.' not in url.split('/')[-1]:
+ url += '/@@index.html'
+ return url
+
+def isLocalURL(url):
+ """Determine whether the passed in URL is local and accessible."""
+ if url.startswith('javascript:'):
+ return False
+ if url.startswith('mailto:'):
+ return False
+ if url.startswith('http://') and not url.startswith(URL):
+ return False
+ return True
+
+
+class StaticAPODoc(object):
+ """Static API doc Maker"""
+
+ def __init__(self):
+ self.linkQueue = [mechanize.Link(URL, START_PAGE, '', '', ())]
+ for url in additionalURLs:
+ self.linkQueue.append(mechanize.Link(URL, url, '', '', ()))
+ self.rootDir = os.path.join(os.path.dirname(__file__), BASE_DIR)
+ self.maxWidth = getMaxWidth()-13
+
+ def start(self):
+ """Start the retrieval of the apidoc."""
+ t0 = time.time()
+ c0 = time.clock()
+
+ self.visited = []
+ self.counter = 0
+
+ if not os.path.exists(self.rootDir):
+ os.mkdir(self.rootDir)
+
+ if USE_PUBLISHER:
+ self.sendMessage('Setting up Zope 3.')
+ functional.FunctionalTestSetup()
+ self.browser = zope.testbrowser.testing.PublisherMechanizeBrowser()
+ self.browser.addheaders.append(
+ ('Authorization', 'Basic mgr:mgrpw'))
+ else:
+ self.browser = mechanize.Browser()
+ self.browser.addheaders.append(
+ ('Authorization', 'Basic Z2FuZGFsZjoxMjM='))
+
+ self.browser.urltags = urltags
+
+ # Work through all links until there are no more to work on.
+ self.sendMessage('Starting retrieval.')
+ while self.linkQueue:
+ link = self.linkQueue.pop()
+ # Sometimes things are placed many times into the queue, for example
+ # if the same link appears twice in a page. In those cases, we can
+ # check at this point whether the URL has been already handled.
+ if link.absolute_url not in self.visited:
+ self.showStatistics(link)
+ self.processLink(link)
+
+ t1 = time.time()
+ c1 = time.clock()
+
+ self.sendMessage(
+ "Run time: %.3f sec real, %.3f sec CPU" %(t1-t0, c1-c0))
+
+ def showStatistics(self, link):
+ self.counter += 1
+ if VERBOSITY >= 3:
+ url = link.absolute_url[-(self.maxWidth):]
+ sys.stdout.write('\r' + ' '*(self.maxWidth+13))
+ sys.stdout.write('\rLink %5d: %s' % (self.counter, url))
+ sys.stdout.flush()
+
+ def sendMessage(self, msg, verbosity=4):
+ if verbosity >= VERBOSITY:
+ sys.stdout.write('\n')
+ sys.stdout.write(VERBOSITY_MAP.get(verbosity, 'INFO')+': ')
+ sys.stdout.write(msg)
+ sys.stdout.write('\n')
+ sys.stdout.flush()
+
+ def processLink(self, link):
+ """Process a link."""
+ url = link.absolute_url
+
+ # Make sure the directory exists and get a file path.
+ relativeURL = url.replace(URL, '')
+ dir = self.rootDir
+ segments = relativeURL.split('/')
+ filename = segments.pop()
+
+ for segment in segments:
+ dir = os.path.join(dir, segment)
+ if not os.path.exists(dir):
+ os.mkdir(dir)
+
+ filepath = os.path.join(dir, filename)
+
+ # Whatever will happen, we have looked at the URL
+ self.visited.append(url)
+
+ # Retrieve the content
+ try:
+ self.browser.open(url)
+ except urllib2.HTTPError:
+ # TODO: Provide less misleading message; many different errors can
+ # happen here.
+ self.sendMessage('Link not found: ' + link.absolute_url, 2)
+ return
+ except ValueError:
+ # We had a bad URL running the publisher browser
+ self.sendMessage('Bad URL: ' + link.absolute_url, 2)
+ return
+
+ response = self.browser.response()
+ old_location = response.tell()
+ response.seek(0)
+
+ if USE_PUBLISHER:
+ # Remove HTTP Headers
+ for line in iter(lambda: response.readline().strip(), ''):
+ pass
+
+ contents = response.read()
+ response.seek(old_location)
+
+ # Now retrieve all links
+ if self.browser.viewing_html():
+
+ try:
+ links = self.browser.links()
+ except HTMLParser.HTMLParseError:
+ self.sendMessage('Failed to parse HTML: ' + url, 1)
+ links = []
+
+ for link in links:
+ # Make sure URLs have file extensions, but no parameters
+ link.url = completeURL(cleanURL(link.url))
+ link.absolute_url = completeURL(cleanURL(link.absolute_url))
+ # Add link to the queue
+ if link.absolute_url not in self.visited:
+ if isLocalURL(link.url):
+ self.linkQueue.insert(0, link)
+
+ # Rewrite URLs
+ if isLocalURL(link.url):
+ parts = ['..']*len(segments)
+ parts.append(link.absolute_url.replace(URL, ''))
+ contents = contents.replace(link.url, '/'.join(parts))
+
+ # Write the data into the file
+ file = open(filepath, 'w')
+ file.write(contents)
+ file.close()
+
+ # Cleanup; this is very important, otherwise we are opening too many
+ # files.
+ self.browser.response().close() # bug fix
+ self.browser.close()
+ self.browser._history = [] # bug fix
+
+
+def main():
+ global BASE_DIR
+ BASE_DIR = sys.argv[1]
+ maker = StaticAPODoc()
+ maker.start()
+ sys.exit(0)
Property changes on: Zope3/trunk/src/zope/app/apidoc/static.py
___________________________________________________________________
Name: svn:eol-style
+ native
Added: Zope3/trunk/utilities/static-apidoc
===================================================================
--- Zope3/trunk/utilities/static-apidoc 2005-10-26 16:44:32 UTC (rev 39639)
+++ Zope3/trunk/utilities/static-apidoc 2005-10-26 16:51:16 UTC (rev 39640)
@@ -0,0 +1,37 @@
+#!/usr/bin/env python2.4
+##############################################################################
+#
+# Copyright (c) 2004 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Script to create a static version of the apidoc tool.
+
+$Id: $
+"""
+import os
+import sys
+
+here = os.path.dirname(os.path.realpath(__file__))
+swhome = os.path.dirname(here)
+
+for parts in [("src",), ("lib", "python"), ("Lib", "site-packages")]:
+ d = os.path.join(swhome, *(parts + ("zope",)))
+ if os.path.isdir(d):
+ d = os.path.join(swhome, *parts)
+ sys.path.insert(0, d)
+ break
+else:
+ print >> sys.stderr, "Could not locate Zope software installation!"
+ sys.exit(1)
+
+
+from zope.app.apidoc.static import main
+main()
Property changes on: Zope3/trunk/utilities/static-apidoc
___________________________________________________________________
Name: svn:executable
+ *
More information about the Zope3-Checkins
mailing list