[Zope-CVS] CVS: Packages/FunctionalTests/doc - CHANGES.txt:1.1 FunctionalTests.odm:1.1 ISSUES.txt:1.1 README.txt:1.1
Tres Seaver
tseaver@zope.com
Thu, 24 Apr 2003 17:39:54 -0400
Update of /cvs-repository/Packages/FunctionalTests/doc
In directory cvs.zope.org:/tmp/cvs-serv3971/doc
Added Files:
CHANGES.txt FunctionalTests.odm ISSUES.txt README.txt
Log Message:
- Repackage for distutils.
=== Added File Packages/FunctionalTests/doc/CHANGES.txt ===
FunctionalTests Changelog
Release 0.4 -- 2003/04/24
- Packaged for distutils.
Release 0.3 -- 2002/04/30
- Framework: Modified Result to propagate cookies through entire
result tree.
- Framework: added 'repeat' directive for both requests and sub-tests.
- Framework: added SleepRequest, to permit injection of "shim" time
between real requests.
Release 0.2 -- 2002/03/14
- FTRunner: added parsing of arbitrary defaults from command-line
as name=value pairs.
- Framework: added ability to capture and return server-originated
cookies (e.g., to allow testing of session-based applications).
Release 0.1 -- 2002/01/??
- Initial release
=== Added File Packages/FunctionalTests/doc/FunctionalTests.odm === (2427/2527 lines abridged)
ObjectDomain_Model 10
{z8kflt_-bfo_cs5fga3q_-p98 0
}[objectdomain.uml.behavioral.collaboration.Collaboration-owns $19
objectdomain.uml.core.TypeExpression $31
objectdomain.uml.behavioral.common.ReturnAction-actionContainer $25
objectdomain.uml.core.UmlTaggedValue-taggedValues $12
objectdomain.uml.view.ClassifierRoleView-owns $8
objectdomain.sg.sgraphics.SGImage $34
objectdomain.uml.core.UmlOperation-features $15
objectdomain.uml.mechanisms.UmlModel-models $1
objectdomain.uml.core.UmlAttribute-features $17
objectdomain.uml.mechanisms.StereotypePackage-owns $26
objectdomain.domain.DomainLockObject $33
objectdomain.uml.behavioral.common.CallAction-actionContainer $23
objectdomain.domain.ModelEnvironment-owns $2
objectdomain.uml.behavioral.common.UmlArgument-arguments $24
objectdomain.uml.view.GeneralizationView-owns $5
objectdomain.uml.behavioral.collaboration.ClassifierRole-owns $20
objectdomain.uml.view.Diagram-owns $3
objectdomain.uml.mechanisms.StylePackage-owns $28
objectdomain.uml.mechanisms.UmlPackage-owns $10
objectdomain.uml.view.MessageView-owns $9
objectdomain.uml.core.UmlExpression $32
objectdomain.uml.core.UmlParameter-parameters $16
objectdomain.uml.core.Projection $29
objectdomain.uml.core.UmlAssociationEnd-connections $14
objectdomain.uml.core.UmlMultiplicity $30
objectdomain.uml.core.Generalization-owns $18
objectdomain.uml.view.AssociationView-owns $6
objectdomain.uml.view.ClassView-owns $4
objectdomain.uml.core.UmlClass-owns $11
objectdomain.uml.core.UmlAssociation-owns $13
objectdomain.uml.behavioral.common.CreateAction-actionContainer $22
objectdomain.uml.core.Stereotype-owns $27
objectdomain.uml.behavioral.collaboration.UmlMessage-owns $21
objectdomain.uml.view.SequenceDiagram-sequenceDiagrams $7
]
($1 1 > z8kflt_-bfo_cs5fga3q_-p98 0
(name "Model"
)
(uid "z8kflt_-bfo_cs5fga3q_-p98")
(namespaceRoot true))
($2 2 < z8kflt_-bfo_cs5fga3q_-p97 "ModelEnvironment" 1
(uid "z8kflt_-bfo_cs5fga3q_-p97"))
($10 47 < z8kflt_-bfo_cs5fga3q_-p7o "Logical" 1
(name "Logical"
)
(uid "z8kflt_-bfo_cs5fga3q_-p7o")
(namespaceRoot true))
($3 3 < z8kflt_-bfo_cs5fga3q_-p7n "Logical diagram" 47
[-=- -=- -=- 2427 lines omitted -=- -=- -=-]
(receiverAssn 357 356)
(successorAssn 357 362)
(senderAssn 357 353)
(projectionAssn 36 357 473)
(to 37 32)
(from 37 31)
(projectionAssn 37 359 476)
(to 38 33)
(from 38 32)
(projectionAssn 38 362 477)
(projectionAssn 39 365 478)
(to 40 34)
(from 40 33)
(projectionAssn 40 366 479)
(to 41 39)
(from 41 34)
(projectionAssn 41 370 480)
(to 42 39)
(from 42 34)
(projectionAssn 42 372 481)
(to 43 35)
(from 43 34)
(projectionAssn 43 379 482)
(to 44 39)
(from 44 34)
(projectionAssn 44 384 483)
(to 45 39)
(from 45 34)
(projectionAssn 45 387 484)
(to 46 31)
(from 46 32)
(projectionAssn 46 390 485)
(openDiagrams 2 30)
(selection 2 11)
(lockedBy 47 486)
(icon 394 487)
(lockedBy 394 486)
(stereotypeAssn 47 394)
(icon 432 488)
(icon 433 489)
(icon 434 490)
(lockedBy 393 486)
(lockedBy 436 486)
(lockedBy 437 486)
(stereotypeAssn 437 394)
(lockedBy 438 486)
(stereotypeAssn 438 394)
(lockedBy 439 486)
(stereotypeAssn 439 394)
(lockedBy 1 486)
=== Added File Packages/FunctionalTests/doc/ISSUES.txt ===
Issues
- "Local" configuration (hostname, port #, site root) is mixed up
with "general" stuff (request definition).
- Running the driver requires having the environment set up "just
right" (correct Python, correct INSTANCE_HOME/SOFTWARE_HOME/PYTHONPATH,
etc.)
- Ideally, the tests would run against an isolated Zope instance,
which served from a DemoStorage, backed by a FileStorage in a known
state (lots of the setup code would go away here).
- Could we "record" scenarios for later replay, perhap using John
Odom's derived HTTPPublisher? Alternately, could we parse the
scripts generated by OpenSTA?
=== Added File Packages/FunctionalTests/doc/README.txt ===
FunctionTests README
What is this?
FunctionalTests is a set of utilities for building automated
functional tests of web applications. Such tests are useful
for several reasons:
- They provide a crisp specification of the
externally-visible "web API" of the application.
- They make delivered functionality visible to the customer
in an objective fashion: if the customer believes that the
tests are sufficient, and the tests all pass, then we are
done.
- They allow for simple instrumentation of the application,
chiefly to permit developers to identify and correct
software errors which lead to grossly-unacceptable scaling
or performance.
- They provide a fairly realistic basis for doing load
testing of a site, as the sequence of requests which
comprise a use case are, by definition, likely to occur
"together" in typical use.
How does it work?
The package structure of functional tests for an application
should follow closely the package structure of its use cases.
Thus, each "use case package" in the system should map to a
"functional testing package", which is a directory, possibly
within a hierarchy.
In an incrementally-delivered application (such as the
typical consulting gig), each increment will have a package
of use cases which it implements, and a corresponding package
of functional tests for the use cases.
Each use case should have one or more functional tests which
exercise it.
Each functional test consists of a set of request
specifications, which if executed successfully (and in order)
demonstrate that the use case is satisfied. In particular,
the functional test specification contains:
o Specification of any preconditions / setup needed to run
the test, expressed as a Python Script / External
Method. The directory for the functional test should
contain the source of this method.
o Specification of the request sequence which fulfills
the use case, including:
+ URL
+ Authentication
+ Any other cookies or HTTP headers
+ POST data, if any
+ PUT content, if any
o Specification of a postcondition for the case again
expressed as a Python Script or External Method; this
method should return an HTTP error response code if the
postcondition is not met. As with the setup method, the
source for the postcondition method should be stored in
the directory containing the functional test.
o Specification of any cleanup required at the close of
the request, expressed as a Python Script or External
Method. Again, the source for this method should be
stored in the directory with the functional test.
Sample Usage
Within the Python debugger::
>>> defaults = { 'portal_url':'http://localhost:8080/functest'
... , 'site_path':'/functest' }
>>> from FunctionalTests.Framework import buildTest
>>> t = buildTest( '/home/tseaver/projects/ZopeOrg'
... '/functional_tests/CMF_base/submit_news.txt'
... , defaults=defaults )
>>> r = t( app=app, defaults=defaults )
>>> assert r()
>>> for invocation in r.listInvocations():
... print invocation.getReplyStatus()
...
200 OK
200 OK
200 OK
200 OK
200 OK
302 Moved Temporarily
302 Moved Temporarily
200 OK
302 Moved Temporarily
200 OK
200 OK
302 Moved Temporarily
File Formats
Packages
For the product being tested, create a 'functional_tests'
subdirectory (like the one this file is in). It should
consist of a README (like this one) and one or more
subdirectories for the various increments (at least one
should contain "baseline" functional tests for Zope and
CMF).
For each increment, create a new subdirectory. Add a README
file, describing the general scope and goals of the
increment, and enumerating the use cases (qualifying as
needed) which it is to implement.
Package Specification Files
Each package (directory) in the functional testing
hierarchy must specify its contents using an "INI-file"
like file (parseable by the standard Python ConfigParser
module). This file consists of a pair of sections:
- '[Package]' contains metadata about the package, and
enumerates the tests and sub-packages which it contains.
- '[DEFAULT]' contains a set of standard substitution
values, which are propagated to each test and
sub-package as global defaults.
Example package specification file::
[Package]
Title: Browsing the site
Status: Active
Tests: browse_news, browse_events
Packages: Folder_contents
[DEFAULT]
portal_url: http://localhost:8080/functest
Scenario Specification Files
For each scenario of a tested use case, create an "INI-file"
like file (one parseable by the standard Python
ConfigParser). The '[Scenario]' section should contain the
title of the scenario, and the name of its use case.
Each request section should contain fields for the URL,
the authentication tokens, a list of form fields, a
list of cookies, and a list of additional headers.
Example scenario specification::
[Scenario]
Title: Browse news
Use_case: Guest explores site.
[Setup]
# adds '/testnews', plus two news items, and makes them published
URL: %(portal_url)s/scaffolding/CMF_base/browse_news_setup
[Home_Page]
URL: %(portal_url)s/
Expected_Result: 200
[News_Page]
URL: %(portal_url)s/recent_news
Expected_Result: 200
[News_Item_1]
URL: %(portal_url)s/testnews/item_1
Expected_Result: 200
[News_Item_2]
URL: %(portal_url)s/testnews/item_2
Expected_Result: 200
[Teardown]
# removes '/testnews'.
URL: %(portal_url)s/scaffolding/CMF_base/browse_news_teardown