[Checkins] SVN: zc.buildout/branches/tlotze-download-api/src/zc/buildout/ - added a custom URLOpener class that doesn't eat exceptions when a download failed

[Thomas Lotze]

Log message for revision 100122:
  - added a custom URLOpener class that doesn't eat exceptions when a download failed
  - added some tests for downloading without using the cache
  

Changed:
  U   zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.py
  A   zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt
  U   zc.buildout/branches/tlotze-download-api/src/zc/buildout/tests.py

-=-
Modified: zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.py
===================================================================
--- zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.py	2009-05-18 23:30:30 UTC (rev 100121)
+++ zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.py	2009-05-19 06:57:43 UTC (rev 100122)
@@ -24,6 +24,13 @@
 import urlparse
 
 
+class URLOpener(urllib.FancyURLopener):
+    http_error_default = urllib.URLopener.http_error_default
+
+
+url_opener = URLOpener()
+
+
 class Download(object):
     """Configurable download utility.
 
@@ -98,6 +105,7 @@
         See __call__.
 
         """
+        urllib._urlopener = url_opener
         path, headers = urllib.urlretrieve(url, path)
         if not check_md5sum(path, md5sum):
             raise ValueError('MD5 checksum mismatch downloading %r' % url)

Added: zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt
===================================================================
--- zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt	                        (rev 0)
+++ zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt	2009-05-19 06:57:43 UTC (rev 100122)
@@ -0,0 +1,52 @@
+==========================
+Using the download utility
+==========================
+
+The ``zc.buildout.download`` module provides a download utility that handles
+the details of downloading files needed for a buildout run from the internet.
+It downloads files to the local file system, using the download cache if
+desired and optionally checking the downloaded files' MD5 checksum.
+
+We setup an HTTP server that provides a few files:
+
+>>> root = tmpdir('sample_files')
+>>> write(join(root, 'foo.txt'), 'This is a foo text.')
+>>> write(join(root, 'bar.cfg'), '[DEFAULTS]\nbar=baz\n')
+>>> server = start_server(root)
+
+Downloading without using the cache
+===================================
+
+If no download cache should be used, the download utility is instantiated
+given buildout's options and switching off the cache, and called to download
+from a URL:
+
+>>> from zc.buildout.download import Download
+>>> download = Download({}, use_cache=False)
+>>> path = download(server+'foo.txt')
+>>> print open(path).read()
+This is a foo text.
+
+As we aren't using the download cache and haven't specified a target path
+either, the download has ended up in a temporary file:
+
+>>> import tempfile
+>>> path.startswith(tempfile.gettempdir())
+True
+
+When trying to access a file that doesn't exist, we'll get an exception:
+
+>>> download(server+'not-there')
+Traceback (most recent call last):
+IOError: ('http error', 404, 'Not Found',
+          <httplib.HTTPMessage instance at 0xa0ffd2c>)
+
+We can also have the downloaded file's MD5 sum checked:
+
+>>> try: from hashlib import md5
+... except ImportError: from md5 import new as md5
+
+>>> path = download(server+'foo.txt', md5('This is a foo text.').hexdigest())
+>>> path = download(server+'foo.txt', md5('The wrong text.').hexdigest())
+Traceback (most recent call last):
+ValueError: MD5 checksum mismatch downloading 'http://localhost/foo.txt'


Property changes on: zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt
___________________________________________________________________
Added: svn:keywords
   + Id Rev Date
Added: svn:eol-style
   + native

Modified: zc.buildout/branches/tlotze-download-api/src/zc/buildout/tests.py
===================================================================
--- zc.buildout/branches/tlotze-download-api/src/zc/buildout/tests.py	2009-05-18 23:30:30 UTC (rev 100121)
+++ zc.buildout/branches/tlotze-download-api/src/zc/buildout/tests.py	2009-05-19 06:57:43 UTC (rev 100122)
@@ -1,6 +1,6 @@
 ##############################################################################
 #
-# Copyright (c) 2004 Zope Corporation and Contributors.
+# Copyright (c) 2004-2009 Zope Corporation and Contributors.
 # All Rights Reserved.
 #
 # This software is subject to the provisions of the Zope Public License,
@@ -2806,6 +2806,20 @@
                (re.compile('[-d]  setuptools-\S+[.]egg'), 'setuptools.egg'),
                ]),
             ),
+
+
+        doctest.DocFileSuite(
+            'download.txt',
+            setUp=easy_install_SetUp,
+            tearDown=zc.buildout.testing.buildoutTearDown,
+            optionflags=doctest.NORMALIZE_WHITESPACE,
+            checker=renormalizing.RENormalizing([
+               (re.compile('0x[0-9a-f]+'), '<MEM ADDRESS>'),
+               (re.compile('http://localhost:[0-9]{4,5}/'),
+                'http://localhost/'),
+               ]),
+            ),
+
         doctest.DocTestSuite(
             setUp=easy_install_SetUp,
             tearDown=zc.buildout.testing.buildoutTearDown,